MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 1 callersMethod_case_projection
Pair raw metrics first, using the current collector's pairing kernel.
src/skillevaluator/tier3/harbor/stats.py:231
↓ 1 callersFunction_case_scored_metrics
The metrics each case scored with a number in at least one attempt.
src/skillevaluator/tier3/harbor/collector.py:5353
↓ 1 callersFunction_case_selection
Return the recorded ``max_cases`` counts so reports can flag a partial run.
src/skillevaluator/evaluation/tier3_report.py:4873
↓ 1 callersFunction_case_skill_metric_reasons
Return the skill metrics this eval case leaves N/A, each with its reason.
src/skillevaluator/tier3/harbor/templates/eval.py:4119
↓ 1 callersFunction_case_span
Compact case label: ``001-003`` for a consecutive run, else a list, else a count.
src/skillevaluator/reporting/console_ui.py:437
↓ 1 callersFunction_case_state
Case glyph: ✓ every attempt passed, ✗ one failed, ⊘ one did not run, ▸ running, · waiting.
src/skillevaluator/reporting/console_ui.py:304
↓ 1 callersFunction_cases_for_result
( result: ValidationResult, workspace_root: Path | None, scan_root: Path | None, target: str =
src/skillevaluator/reporting/junit_reporter.py:271
↓ 1 callersMethod_catalog_display_name
Return a result-safe catalog label without host directory details.
src/skillevaluator/validators/similarity.py:260
↓ 1 callersFunction_catalog_entry_with_metadata_id_conflicts
Project catalog findings onto a member, retaining its independent report verdict. The findings themselves are listed under ``catalog_findings`` s
src/skillevaluator/cli.py:1614
↓ 1 callersFunction_catalog_line
(rows: list[dict[str, Any]])
src/skillevaluator/harness/report.py:114
↓ 1 callersFunction_catalog_listing_is_authoritative
Return whether absence from the listing proves the model is unusable.
src/skillevaluator/tier3/harbor/runtime_preflight.py:741
↓ 1 callersFunction_catalog_of
(group: list[Path])
src/skillevaluator/harness/overlay.py:658
↓ 1 callersFunction_catalog_roots
(root: Path, *, max_entries: int)
src/skillevaluator/deduplication/plugin/catalog.py:354
↓ 1 callersFunction_catalog_skips
(reason)
src/skillevaluator/deduplication/plugin/commands.py:182
↓ 1 callersFunction_cell
(record: dict | None, skill: dict | None, *, expected: bool, path: str | None = None)
ci/comment_summary.py:68
↓ 1 callersFunction_cell_text
(status: str, issues: str, reason: str = "")
ci/comment_summary.py:61
↓ 1 callersFunction_census_label
*value* redacted whole, then made a label of at most *limit* characters. ``_safe_text`` redacts only a window of the raw text, and a token the wi
src/skillevaluator/tier3/eval_core/runtime_evidence.py:73
↓ 1 callersFunction_census_load
What the load census says about each MCP server, over every agent. Returns ``(servers some agent's harness loaded in every trial, server -> w
src/skillevaluator/tier3/mcp_proof.py:838
↓ 1 callersFunction_change_cell
(delta: object)
src/skillevaluator/reporting/tier3_token_usage.py:103
↓ 1 callersFunction_changelog_section
The heading of the CHANGELOG section that holds the entry starting with these words.
tests/tier3/test_dataset_baseline_fairness.py:1227
↓ 1 callersFunction_chat_completion_payload
( model, prompt, max_tokens, temperature, provider=None, request_url=None, respons
src/skillevaluator/tier3/harbor/templates/eval.py:1434
↓ 1 callersFunction_chat_custom_tool_input
(arguments: str)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1912
↓ 1 callersFunction_chat_enabled
(requested: bool | None)
src/skillevaluator/tier_workflows.py:32
↓ 1 callersMethod_checkEventBindings
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunction_check_ack_cluster_readiness_subprocess
Run ACK's bounded pod-list probe under Harbor's exact child environment.
src/skillevaluator/tier3/harbor/runner.py:2489
↓ 1 callersFunction_check_arg_mapping
(value: Any, where: str, errors: _FieldErrors, *, kind: str)
src/skillevaluator/tier3/eval_core/plugin_signals.py:924
↓ 1 callersFunction_check_claude_channels
(value: Any, issues: list[ManifestIssue], *, label: str)
src/skillevaluator/plugin_format_schema.py:822
↓ 1 callersFunction_check_claude_commands
``commands``: a path, an array of paths, or a map of command names to command objects.
src/skillevaluator/plugin_format_schema.py:843
↓ 1 callersFunction_check_claude_experimental
(value: Any, issues: list[ManifestIssue], *, label: str)
src/skillevaluator/plugin_format_schema.py:869
↓ 1 callersFunction_check_claude_lsp_servers
``lspServers``: a path, a server map, or an array of those; every inline server config is strict.
src/skillevaluator/plugin_format_schema.py:680
↓ 1 callersFunction_check_claude_name
(data: dict[str, Any], issues: list[ManifestIssue], *, label: str)
src/skillevaluator/plugin_format_schema.py:388
↓ 1 callersMethod_check_code_error_handling
(dim, files: list[Path], rule: CodeErrorHandlingRule)
src/skillevaluator/validators/quality_score.py:835
↓ 1 callersFunction_check_codex_mcp_toml
Parse the generated Codex MCP tables now, so a bad config fails staging, not every Codex trial.
src/skillevaluator/tier3/plugin_native.py:1762
↓ 1 callersFunction_check_codex_name
(data: dict[str, Any], issues: list[ManifestIssue], *, label: str)
src/skillevaluator/plugin_format_schema.py:1390
↓ 1 callersFunction_check_codex_version
(data: dict[str, Any], issues: list[ManifestIssue], *, label: str)
src/skillevaluator/plugin_format_schema.py:1429
↓ 1 callersMethod_check_correctness
( self, qs: QualityScoreResult, content: str, skill_path: Path, frontm
src/skillevaluator/validators/quality_score.py:494
↓ 1 callersFunction_check_cursor_references
LOW: Cursor's ``${env:NAME}`` is a reference (never an inline secret), but its expansion is unverified.
src/skillevaluator/validators/mcp_static.py:3547
↓ 1 callersFunction_check_custom_environment_does_not_stage_target
(custom_env_dir: Path, target_skill: Path)
src/skillevaluator/tier3/harbor/adapter.py:5735
↓ 1 callersFunction_check_dataset
Check for the required dataset file. Returns True if found.
src/skillevaluator/tier3/evals_spec.py:599
↓ 1 callersMethod_check_discoverability
( self, qs: QualityScoreResult, content: str, frontmatter: dict | None, )
src/skillevaluator/validators/quality_score.py:713
↓ 1 callersMethod_check_efficiency
( self, qs: QualityScoreResult, content: str, lines: list[str], skill_
src/skillevaluator/validators/quality_score.py:854
↓ 1 callersFunction_check_endpoint
Report a metadata or non-allowlisted private host of one URL field (``label``), shown as ``shown``.
src/skillevaluator/validators/mcp_static.py:3049
↓ 1 callersFunction_check_env_and_headers
(server: _ServerFindings, config: dict[str, Any], *, client: McpClient)
src/skillevaluator/validators/mcp_static.py:1134
↓ 1 callersFunction_check_expected_source
Reject a card that does not name the source the publishing pipeline requested. This applies to every verdict: a FAIL card for another commit is a
scripts/ci/check_public_benchmarks.py:1962
↓ 1 callersMethod_check_frontmatter
Tier 1: Extract license from YAML frontmatter.
src/skillevaluator/validators/license.py:317
↓ 1 callersMethod_check_frontmatter_correctness
Validate frontmatter fields that go beyond basic SchemaValidator checks.
src/skillevaluator/validators/quality_score.py:555
↓ 1 callersFunction_check_html_rendering_safety
Reject raw HTML whose browser presentation cannot be parsed safely.
scripts/ci/check_public_benchmarks.py:951
↓ 1 callersFunction_check_insecure_tls_config
Reject config keys that turn off TLS/certificate verification.
src/skillevaluator/validators/mcp_static.py:1272
↓ 1 callersFunction_check_literal_scheme
The scheme rules for a URL whose host or port comes from a reference, when its scheme is written out.
src/skillevaluator/validators/mcp_static.py:967
↓ 1 callersFunction_check_metadata_semantics
( path: Path, text: str, offenders: list[Offender], *, require_source_provenance: bool = F
scripts/ci/check_public_benchmarks.py:994
↓ 1 callersFunction_check_native_source_does_not_stage_target
( native_dir: Path, target_skill: Path, *, with_skill: bool, )
src/skillevaluator/tier3/harbor/adapter.py:5762
↓ 1 callersFunction_check_oauth_urls
Claude Code fetches OAuth server metadata from ``oauth.authServerMetadataUrl``: the same URL policy applies.
src/skillevaluator/validators/mcp_static.py:981
↓ 1 callersFunction_check_overrides
Agent-CLI permission flags anywhere in the entry (bypass flags, permissive modes such as ``--permission-mode acceptEdits``, and ``--allowedTools``
src/skillevaluator/validators/mcp_static.py:3505
↓ 1 callersFunction_check_pass_provenance
Reject PASS cards that replace required provenance with legacy placeholders.
scripts/ci/check_public_benchmarks.py:2004
↓ 1 callersFunction_check_pass_source_identity
Require every PASS card to identify the exact evaluated source tree.
scripts/ci/check_public_benchmarks.py:1940
↓ 1 callersMethod_check_reliability
( self, qs: QualityScoreResult, content: str, skill_path: Path, )
src/skillevaluator/validators/quality_score.py:805
↓ 1 callersFunction_check_required_headings
Require canonical headings in rendered Markdown structure.
scripts/ci/check_public_benchmarks.py:902
↓ 1 callersFunction_check_schema
(schema: Any, where: str, errors: _FieldErrors, *, depth: int, budget: list[int])
src/skillevaluator/tier3/eval_core/plugin_signals.py:854
↓ 1 callersFunction_check_score_transitions
Check the score transitions in Results at a Glance table body cells. The reporter writes only its own numbers into these cells, so untrusted
scripts/ci/check_public_benchmarks.py:859
↓ 1 callersFunction_check_secret_values
Inline credentials (and, for env and headers, TLS-off switches) among one block's string values.
src/skillevaluator/validators/mcp_static.py:1183
↓ 1 callersFunction_check_selection_overlap
A decoy ref must not also name an expected or acceptable call: it would be scored both ways.
src/skillevaluator/tier3/eval_core/plugin_signals.py:1212
↓ 1 callersFunction_check_source_provenance
Reject a published PASS that does not record the source it evaluated.
scripts/ci/check_public_benchmarks.py:1951
↓ 1 callersMethod_check_spec_fields
(self, qs: QualityScoreResult, frontmatter: dict | None)
src/skillevaluator/validators/quality_score.py:978
↓ 1 callersFunction_check_transport
(server: _ServerFindings, config: dict[str, Any])
src/skillevaluator/validators/mcp_static.py:1217
↓ 1 callersMethod_check_type_specific
Apply type-specific correctness checks.
src/skillevaluator/validators/quality_score.py:610
↓ 1 callersFunction_check_unrecognised
Warn about files in evals/ that the toolchain doesn't recognise.
src/skillevaluator/tier3/evals_spec.py:1296
↓ 1 callersFunction_check_untrusted_structural_headings
Require root h1/h2 structure to use unambiguous plain Markdown text.
scripts/ci/check_public_benchmarks.py:943
↓ 1 callersFunction_check_url
( server: _ServerFindings, config: dict[str, Any], allowed_private_hosts: HostAllowlist, *, client: McpCli
src/skillevaluator/validators/mcp_static.py:993
↓ 1 callersFunction_check_verdict_tier_consistency
( path: Path, text: str, offenders: list[Offender], *, require_source_provenance: bool = F
scripts/ci/check_public_benchmarks.py:1866
↓ 1 callersMethod_child_skill_manifests
The ``SKILL.md`` of each skill folder directly in a declared skills dir. A linked child folder or skill manifest is reported and not followed
src/skillevaluator/plugin_discovery.py:762
↓ 1 callersMethod_choose_from_license_files
Pick a file detection, failing closed on blocked or conflicting ids.
src/skillevaluator/validators/license.py:488
↓ 1 callersFunction_choose_shared_texts
Return the repeated texts to share, mapped to their block numbers in order. A reference to block 10 or later is one character longer than one to
src/skillevaluator/deduplication/intra_skill/llm_analyzer.py:95
↓ 1 callersFunction_chunk_is_short_config
Return True if a chunk is a short block dominated by comment/config-style lines. The chunk's own heading line and code-fence markers are ignored;
src/skillevaluator/deduplication/intra_skill/intra_skill_validator.py:89
↓ 1 callersMethod_circumference
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunction_claim_is_grounded
( item: dict, grounding: tuple[dict[str, bool | None], dict[str, str]], )
src/skillevaluator/evaluation/insights_judge.py:502
↓ 1 callersFunction_claim_text
(item: dict)
src/skillevaluator/evaluation/insights_judge.py:468
↓ 1 callersFunction_claimable_results
The step results this call may claim: an unwrapped inner call only when the normalizer proved it owns them.
src/skillevaluator/tier3/eval_core/plugin_signals.py:2945
↓ 1 callersFunction_clamped
Clamp a desired limit to the current hard limit (soft == hard).
src/skillevaluator/tier3/harbor/local_sandbox.py:347
↓ 1 callersMethod_class
(self)
src/skillevaluator/plugin_component_risk.py:1383
↓ 1 callersMethod_class_atom
(self)
src/skillevaluator/plugin_component_risk.py:1406
↓ 1 callersFunction_class_item
(atom: int | _ClassItem)
src/skillevaluator/plugin_component_risk.py:1246
↓ 1 callersFunction_class_matches
(node: _Node, code: int)
src/skillevaluator/plugin_component_risk.py:1430
↓ 1 callersMethod_classify_by_run_and_category
Classify a finding by consecutive run length and character category.
src/skillevaluator/validators/unicode_smuggle.py:321
↓ 1 callersMethod_classify_char
Classify a character against the lookup tables. Returns (category_name, char_name) or None.
src/skillevaluator/validators/unicode_smuggle.py:215
↓ 1 callersFunction_classify_deno_module
Classify the module ``deno run`` runs: an ``npm:`` or ``jsr:`` package, a remote module, or a local script.
src/skillevaluator/validators/mcp_static.py:2430
↓ 1 callersFunction_classify_execution_failure
( command: str, observation: str, expected_script: str | None, context: dict[str, Any] | None )
src/skillevaluator/tier3/eval_core/checks.py:7284
↓ 1 callersFunction_classify_go_module
``go run module/path@version``: pinned for an exact ``vX.Y.Z``; a local package is not applicable.
src/skillevaluator/validators/mcp_static.py:2457
↓ 1 callersFunction_classify_invocation
(invocation: RunnerInvocation)
src/skillevaluator/validators/mcp_static.py:2544
↓ 1 callersFunction_classify_launch
(config: Any)
src/skillevaluator/validators/mcp_static.py:2649
↓ 1 callersFunction_classify_npm_spec
Classify an npm package spec (``pkg``, ``@scope/pkg@1.2.3``, git/URL, local path).
src/skillevaluator/validators/mcp_static.py:2370
↓ 1 callersFunction_classify_nuget_spec
A .NET tool as ``dnx`` runs it: ``Package`` or ``Package@version``.
src/skillevaluator/validators/mcp_static.py:2469
↓ 1 callersFunction_classify_python_invocation
``uvx``, ``uv tool run``, ``uv run``, and ``pipx run``: the tool spec and every ``--with`` requirement. ``--with`` installs more packages next to
src/skillevaluator/validators/mcp_static.py:2519
↓ 1 callersFunction_classify_python_spec
Classify a PyPI requirement spec as used by ``uvx`` / ``pipx run``.
src/skillevaluator/validators/mcp_static.py:2401
↓ 1 callersFunction_classify_spec_list
The worst classification of several specs: floating, then unpinned, then pinned.
src/skillevaluator/validators/mcp_static.py:2505
↓ 1 callersFunction_claude_dependency_problem
(entry: Any)
src/skillevaluator/plugin_format_schema.py:611
↓ 1 callersFunction_claude_hook_handler
( handler: Any, *, foreign_root: re.Pattern[str] | None = None, plugin_file: Callable[[str], b
src/skillevaluator/tier3/plugin_native.py:583
↓ 1 callersFunction_claude_ignored_values
Claude Code skill fields whose value it ignores, each with what it accepts.
src/skillevaluator/harness/overlay.py:409
↓ 1 callersFunction_claude_lsp_problems
Why Claude Code rejects one LSP server config (a strict object), as short phrases.
src/skillevaluator/plugin_format_schema.py:626
↓ 1 callersFunction_claude_monitor_problems
(item: Any)
src/skillevaluator/plugin_format_schema.py:719
↓ 1 callersFunction_claude_name_refused_by_path
Whether Claude Code refuses this name when it loads the plugin by path (``--plugin-dir``). The rule matches '@', ':', '/', '\\', any whitespace,
src/skillevaluator/plugin_format_schema.py:374
← previousnext →5,801–5,900 of 24,741, ranked by callers