MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 2 callersFunction_job_judge_override
Return the selected dedicated host override name and value, if enabled.
src/skillevaluator/tier3/harbor/runner.py:3107
↓ 2 callersFunction_job_judge_subprocess_env
Make both aliases resolvable while Harbor constructs verifier environments.
src/skillevaluator/tier3/harbor/runner.py:3136
↓ 2 callersFunction_job_judge_verifier_env
Return placeholder-based judge overrides for Harbor's verifier job layer.
src/skillevaluator/tier3/harbor/runner.py:3117
↓ 2 callersFunction_job_passed
Use collector-authoritative logical-attempt semantics for early stop.
src/skillevaluator/tier3/harbor/runner.py:4256
↓ 2 callersFunction_job_stats_match_trials
Require a terminal native job whose counters agree with every physical trial.
src/skillevaluator/tier3/harbor/collector.py:6804
↓ 2 callersFunction_js_name
Whether JavaScript accepts ``name``: `\\w` also takes U+00B2, which it rejects.
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:274
↓ 2 callersFunction_json
(payload: bytes)
ci/publish_reports.py:111
↓ 2 callersFunction_json
(payload: dict, exit_code: int)
tests/validators/test_private_claude_parity.py:37
↓ 2 callersFunction_json_equal
(left: Any, right: Any)
src/skillevaluator/tier3/eval_core/plugin_signals.py:3814
↓ 2 callersFunction_json_safe_agent_eval
Bound and normalize Tier 3 metadata before machine-readable embedding.
src/skillevaluator/reporting/json_reporter.py:55
↓ 2 callersFunction_judge_policies
Return the set judge policies keyed for ``run_config.json``, and an error for each unknown value. Values are case-insensitive and an empty value
src/skillevaluator/tier3/harbor/runner.py:3057
↓ 2 callersFunction_judge_policy
Read one judge policy variable: unset or empty selects ``choices[0]``; another value raises.
src/skillevaluator/tier3/harbor/templates/eval.py:929
↓ 2 callersFunction_judge_token_key
(provider: str, request_url: str)
tests/test_llm_judge.py:280
↓ 2 callersFunction_judged_reward
(case_id: str, accuracy: float, judge_model: str)
tests/test_harbor_metrics_truth.py:692
↓ 2 callersFunction_jwt_fixture
Build a representative JWT at runtime so this test file remains scannable.
tests/validators/test_security.py:107
↓ 2 callersFunction_keep_attached_descriptors
Quote ``2>`` so the lexer keeps the descriptor with its operator. The lexer splits every ``>`` and ``<`` into its own token, so ``2>/dev/null``
src/skillevaluator/tier3/eval_core/shell_lexing.py:136
↓ 2 callersMethod_kept_by_plugin_copy
Whether the plugin arm's whole-plugin copy keeps ``rel`` (same ignore rules as ``stage_native_bundle``).
src/skillevaluator/tier3/loose_parts.py:1215
↓ 2 callersFunction_kept_text_places
Return where each piece of *kept*, the text a masker kept between its markers, starts at the earliest and latest. The pieces are placed in order
src/skillevaluator/tier3/eval_core/checks.py:4134
↓ 2 callersFunction_key_words
Lower-case words of an env name, header, flag, or camelCase key (``clientSecret`` -> client, secret).
src/skillevaluator/validators/url_policy.py:357
↓ 2 callersFunction_kill_group
()
src/skillevaluator/tier3/nemo_gym/runner.py:637
↓ 2 callersFunction_known_route
Return only a known route so logs cannot include query strings or secrets.
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:318
↓ 2 callersFunction_large_run_info
(*, cases, attempts, checks)
tests/reporting/test_declarative_coverage.py:408
↓ 2 callersMethod_launch_args
(self)
src/skillevaluator/utils/tool_runner.py:258
↓ 2 callersFunction_launch_path
The path part of a launch value: the value of a ``--flag=path`` argument, else the value itself.
src/skillevaluator/plugin_mcp.py:464
↓ 2 callersFunction_leaks_host_path
(text: str, path: Path)
tests/test_openclaw_plugin_smoke.py:33
↓ 2 callersFunction_legacy_benchmark
Return a card in the layout rendered before the publication identity fields existed.
tests/test_public_benchmark_gate.py:116
↓ 2 callersFunction_legacy_run_agents
Return configured result-agent keys from an authentic historical config.
src/skillevaluator/tier3/results_location.py:402
↓ 2 callersFunction_lexical_path_components
Normalize separators and dot segments without touching the filesystem.
src/skillevaluator/tier3/eval_core/checks.py:1237
↓ 2 callersFunction_lexical_plugin_path
A plugin-root-relative path with ``.`` and ``..`` resolved lexically; ``None`` when it leaves the root. ``scripts/../node_modules/srv/index.js``
src/skillevaluator/plugin_mcp.py:470
↓ 2 callersMethod_license_policy
Return the policy status of a parsed expression and the name that decided it. Any blocked name blocks the expression, even one that is not va
src/skillevaluator/validators/license.py:421
↓ 2 callersFunction_lift_overall_scores
Return the two overall scores that the published overall lift subtracts. When a metric is N/A in one arm it drops out of both arms for lift, so t
src/skillevaluator/evaluation/tier3_report.py:4591
↓ 2 callersFunction_limit_staged_cases
Remove staged cases after the first ``max_cases`` and describe the selection. Harbor discovers local dataset tasks from their directories, so pru
src/skillevaluator/tier3/harbor/runner.py:3512
↓ 2 callersFunction_line
(value: object)
src/skillevaluator/skillspector_references.py:170
↓ 2 callersFunction_link_display
Keep untrusted link diagnostics bounded and on one physical line.
src/skillevaluator/validators/hygiene.py:23
↓ 2 callersFunction_link_location_parts
(path: Path)
src/skillevaluator/harness/overlay.py:138
↓ 2 callersFunction_lint_findings
(results: Sequence[ValidationResult])
src/skillevaluator/precommit.py:167
↓ 2 callersMethod_lint_skill
Lint all Python scripts in a skill's scripts/ or tools/ directory.
src/skillevaluator/validators/script_lint.py:129
↓ 2 callersFunction_listing
(api, path, *, key=None)
ci/feedback.py:251
↓ 2 callersFunction_literal_rm_operands
Return the operands of one literal, recursive and forced ``rm``, else ``None``. ``None`` means *text* is not a single plain rm: it carries shell
src/skillevaluator/tier3/eval_core/checks.py:292
↓ 2 callersFunction_live_console
A Rich console the live view will draw on. ``color_system=None`` keeps the test from caching a colour code on Rich's shared Style objects, wh
tests/reporting/test_tier3_validate_cases.py:27
↓ 2 callersFunction_live_mark
``None`` while the trial runs, else its attempt mark.
src/skillevaluator/tier3/case_summary.py:1603
↓ 2 callersFunction_load
(path: Path | None)
src/skillevaluator/quality_config.py:327
↓ 2 callersFunction_load
(path: Path)
ci/smoke_image.py:178
↓ 2 callersFunction_load_evals
Load normalized dataset entries from evals.json/jsonl/yaml.
src/skillevaluator/tier3/harbor/adapter.py:1750
↓ 2 callersFunction_load_json
(path: Path)
src/skillevaluator/tier3/harbor/templates/custom_grader_runner.py:151
↓ 2 callersFunction_load_mcp_file
Collect the servers of one MCP config file; a file that cannot be used is a broken source. Its finding names the declared ref (``plugin_component
src/skillevaluator/plugin_mcp.py:921
↓ 2 callersFunction_load_native
(path: Path)
src/skillevaluator/reporting/dashboard_data.py:644
↓ 2 callersFunction_load_policy_yaml
Load policy YAML and normalize parser failures to the public contract.
src/skillevaluator/validators/policy.py:192
↓ 2 callersFunction_load_proof
(tally)
tests/tier3/test_private_evidence_review2.py:225
↓ 2 callersFunction_load_tally
(records: tuple[LogicalTrialEvidence, ...])
src/skillevaluator/tier3/plugin_evidence.py:183
↓ 2 callersMethod_locate
(self, relative: str)
src/skillevaluator/validators/pii_waivers.py:280
↓ 2 callersMethod_location_within
A dataset location starts within ``limit`` characters of the noun, in the same phrase.
src/skillevaluator/evaluation/dataset_case_removal.py:313
↓ 2 callersMethod_log
(self, status: int, route: str, category: str)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:309
↓ 2 callersFunction_logical_condition_rewards
Return each arm's logical attempts as the lift rebuild compares them, and whether both loads are complete. The first list holds the with-skill at
src/skillevaluator/evaluation/tier3_report.py:2129
↓ 2 callersFunction_long_file_write
(body_lines: int = 20_000, tail: str = "", header: str | None = None)
tests/tier3/test_issue192_security_false_positives.py:73
↓ 2 callersFunction_looks_like_path_token
(token: str)
src/skillevaluator/tier3/harbor/local_environment.py:297
↓ 2 callersFunction_looks_like_refusal
(agent_text: str)
src/skillevaluator/tier3/eval_core/checks.py:5367
↓ 2 callersFunction_lstat
``(metadata, None)``, ``(None, None)`` when the path is not there, or ``(None, error)``.
src/skillevaluator/plugin_dependencies.py:430
↓ 2 callersFunction_lstat_candidate
(root: Path, relative: Path)
src/skillevaluator/plugin_manifest.py:427
↓ 2 callersFunction_managed_run
Build an exec-form RUN that remains usable after an authored PATH override.
src/skillevaluator/tier3/harbor/adapter.py:3581
↓ 2 callersFunction_manifest_digest
Digest securely captured manifest entries using recipe version 2.
src/skillevaluator/publication_identity.py:587
↓ 2 callersFunction_manifest_reads
The declared members whose ``SKILL.md`` this call reads, in order (every one in a chained shell command).
src/skillevaluator/tier3/eval_core/plugin_signals.py:2654
↓ 2 callersFunction_manifests_match
(first: Iterable[_PublicationEntry], second: Iterable[_PublicationEntry])
src/skillevaluator/publication_identity.py:376
↓ 2 callersFunction_mapping
(value: Any)
src/skillevaluator/evaluation/tier3_expectations.py:29
↓ 2 callersMethod_mark_item
(self)
src/skillevaluator/utils/structured_data.py:316
↓ 2 callersFunction_mark_quoted_syntax
Prefix each quoted or escaped word that would otherwise read as syntax. Words are delimited as the shell delimits them: at unquoted whitespace
src/skillevaluator/tier3/eval_core/shell_lexing.py:67
↓ 2 callersFunction_markdown
( payload: dict, *, publication: bool = False, title: str = "Plugin static evidence", noti
src/skillevaluator/reporting/plugin_static_render.py:23
↓ 2 callersMethod_markdown_dir
Every agent, command, or output style a client loads from a folder, at any depth. Nothing is pruned: Claude Code also loads ``agents/evals/x.
src/skillevaluator/plugin_discovery.py:984
↓ 2 callersMethod_markdown_file
(self, component_type: str, rel: PurePosixPath, origin: Origin)
src/skillevaluator/plugin_discovery.py:1321
↓ 2 callersFunction_markdown_untrusted_table_cell
Return one inert table cell without double-escaping HTML entities.
src/skillevaluator/reporting/markdown.py:145
↓ 2 callersFunction_marked
(marker: str, variants: Iterable[str])
src/skillevaluator/validators/skillspector_scope.py:1115
↓ 2 callersFunction_marker_payload
(destination: Path, key: bytes)
src/skillevaluator/tier3/output_provenance.py:531
↓ 2 callersFunction_marks
(with_skill, without)
tests/reporting/test_tier3_validate_cases.py:42
↓ 2 callersFunction_marks_text
(marks: list[str])
src/skillevaluator/tier3/case_summary.py:464
↓ 2 callersFunction_masker_reading
Return *text* as the masker reads it, without shell line continuations, and where each of its characters is. The positions are None when *text* h
src/skillevaluator/tier3/eval_core/checks.py:4078
↓ 2 callersFunction_masks_only_after_markers
Whether *twice* is *once* with, at most, the text right after some of its markers masked into them. The text before the first marker is the same,
tests/tier3/test_security_finding_evidence.py:4358
↓ 2 callersFunction_match_names
The non-empty ``values``, casefolded: refs match case-insensitively.
src/skillevaluator/tier3/eval_core/plugin_signals.py:3445
↓ 2 callersFunction_matches_error_sequence
Match whole structured errors, including a prefix saved before later errors. Delimiters inside an error are its original prose, never new error r
src/skillevaluator/reporting/insights_view.py:74
↓ 2 callersFunction_matches_generated_name
Match an exclusion name using the filesystem's case semantics.
src/skillevaluator/publication_identity.py:129
↓ 2 callersFunction_matching_case
Map an attempt to its case when exactly one case matches. Every attempt is resolved against the dataset's IDs, scored or not: a scored attemp
src/skillevaluator/tier3/token_usage.py:173
↓ 2 callersFunction_matrix_labels
(html: str, agents: list[str], best_agent: str)
tests/reporting/test_tier3_unavailable_lift_labels.py:63
↓ 2 callersFunction_mcp_entry_claude
One Claude Code ``.mcp.json`` entry: launch fields redacted like the wrapper TOML. Claude Code applies a server's ``env`` (stdio) and ``headers``
src/skillevaluator/tier3/plugin_native.py:1252
↓ 2 callersFunction_mcp_ref_candidates
``<server>`` and ``<server>/<tool>``/``<server>__<tool>``, in the declared and the harness spelling. Claude Code writes the declared ``docs.v2``
src/skillevaluator/tier3/eval_core/plugin_signals.py:3476
↓ 2 callersFunction_mcp_rows
(section: _Section, raw: object, *, nested: bool)
src/skillevaluator/reporting/native_evidence.py:179
↓ 2 callersFunction_md
(value: object)
ci/report_views.py:45
↓ 2 callersFunction_md_links
(links: list[tuple[str, str]])
ci/report_views.py:182
↓ 2 callersFunction_mean
(values: list[float])
src/skillevaluator/evaluation/tier3_report.py:5118
↓ 2 callersFunction_mean_score
(case: Mapping[str, Any] | None)
src/skillevaluator/tier3/case_summary.py:472
↓ 2 callersFunction_meaningful_lines
Non-blank lines of a bounded window, past a harness's status lines (``Wall time:``, ``Output:``).
src/skillevaluator/tier3/eval_core/plugin_signals.py:1794
↓ 2 callersFunction_merge_chain
Return frontmatter whose merge keys repeat earlier mappings at every level.
tests/embedding/test_extractor.py:686
↓ 2 callersFunction_merge_codex_openai_compatible_config
Apply SkillEvaluator's Responses routing without discarding user config.
src/skillevaluator/tier3/harbor/local_agents.py:94
↓ 2 callersFunction_merge_constituent_default_reward_failure
Make an aggregate unscoreable when one of its constituents is invalid.
src/skillevaluator/tier3/harbor/collector.py:3528
↓ 2 callersFunction_merge_harbor_viewer_summaries
(*summaries: dict[str, Any] | None)
src/skillevaluator/evaluation/tier3_report.py:4045
↓ 2 callersFunction_merge_launch_execution_errors
Overlay launch diagnostics without discarding hidden collector counts.
src/skillevaluator/tier3/harbor/runner.py:752
↓ 2 callersFunction_message
(item: Any, file: str | None, roots: tuple[str, ...] = ())
src/skillevaluator/validators/claude_plugin_validate.py:250
↓ 2 callersFunction_metadata_list_items
Return one-line items from top-level unordered metadata lists.
scripts/ci/check_public_benchmarks.py:1531
↓ 2 callersFunction_metric_detail
Return the grader detail that explains one metric score in ``reward``.
src/skillevaluator/evaluation/tier3_report.py:3013
↓ 2 callersFunction_metric_name_contains_sensitive_data
(text: str)
src/skillevaluator/tier3/harbor/templates/custom_grader_runner.py:188
↓ 2 callersFunction_metric_score
(reward: dict[str, Any], metric: str)
src/skillevaluator/tier3/harbor/report.py:263
↓ 2 callersFunction_minimal_roots
Drop paths already covered by an earlier directory exception.
src/skillevaluator/tier3/harbor/local_sandbox.py:692
← previousnext →4,101–4,200 of 24,741, ranked by callers