MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 3 callersFunction_persisted_trial_layout
Allocate portable output names across scored and unscored physical trials.
src/skillevaluator/tier3/harbor/collector.py:5936
↓ 3 callersFunction_phrase
(artifacts: int, occurrences: str)
tests/reporting/test_skillspector_reference_summary.py:93
↓ 3 callersFunction_pick_primary_script
Pick the best script for eval: a script the agent runs, preferably a non-interactive one. Uses SKILL.md context to detect interactive scripts (e.
src/skillevaluator/tier3/generate_dataset.py:310
↓ 3 callersFunction_pipeline_start
Index where the pipeline holding Harbor's launch begins, after Harbor's own setup commands. The setup group goes there, so the commands before th
src/skillevaluator/tier3/harbor/native_agents.py:78
↓ 3 callersFunction_plain_output_env
Keep the real consoles out of terminal mode, so Rich adds no escape sequences of its own.
tests/reporting/test_rich_markup_escaping.py:91
↓ 3 callersFunction_plugin_root_path
Return ``(root-relative path, escapes)`` for a ``${CLAUDE_PLUGIN_ROOT}/...`` (or *refs*) token.
src/skillevaluator/plugin_component_risk.py:1729
↓ 3 callersFunction_plugin_with_dataset
(root: Path)
tests/tier3/test_private_runtime_regressions.py:156
↓ 3 callersFunction_png_chunk
Return one PNG chunk with its length and CRC.
tests/validators/test_security.py:8855
↓ 3 callersFunction_positive_start_line
Return a SARIF-valid positive integer line number, else ``None``.
src/skillevaluator/reporting/sarif_reporter.py:76
↓ 3 callersFunction_pre_agent_setup_command
(pre_agent_setup: list[str] | None)
src/skillevaluator/tier3/harbor/adapter.py:2320
↓ 3 callersFunction_prefixed
(prefix: str, pin: McpPinning)
src/skillevaluator/validators/mcp_static.py:2679
↓ 3 callersFunction_preflight_nemo_gym
Reject options and config the NeMo Gym backend cannot honor, before the dataset step.
src/skillevaluator/tier3/workflow.py:331
↓ 3 callersFunction_preflight_options
Reject known configuration errors before generating a paid dataset. ``harbor_runtime=False`` keeps the backend-neutral checks but skips the Harbo
src/skillevaluator/tier3/workflow.py:106
↓ 3 callersFunction_prepared
(root: Path)
tests/tier3/test_private_runtime_report_acceptance.py:30
↓ 3 callersMethod_pretend_absolute_paths_exist
Model Linux runtime layouts that must not be created on the test host.
tests/test_harbor_local_sandbox.py:342
↓ 3 callersMethod_print_harbor_artifacts
(agent_eval: dict, console: Console)
src/skillevaluator/reporting/cli.py:440
↓ 3 callersFunction_private_runtime_hooks
The private plugin runtime hooks of this run, or ``None`` for an ordinary run.
src/skillevaluator/tier3/harbor/runner.py:5096
↓ 3 callersFunction_private_sandbox_name
Build the retired private label without embedding it in public source.
tests/test_public_benchmark_gate.py:70
↓ 3 callersMethod_process_pip_audit_result
Accept a pip-audit run only when its process state can be trusted.
src/skillevaluator/validators/dependencies.py:552
↓ 3 callersMethod_progress_event
(self, index, state, detail=None, *, stage=None)
src/skillevaluator/reporting/console_ui.py:676
↓ 3 callersFunction_provider
(name: str)
tests/test_custom_execution_backends.py:523
↓ 3 callersFunction_provider_agent_import_path
Select only the provider-specific wrappers required for this environment.
src/skillevaluator/tier3/harbor/runner.py:3693
↓ 3 callersFunction_public_pass_summary
Remove collector-only pairing state from a pass summary.
src/skillevaluator/tier3/harbor/collector.py:5093
↓ 3 callersFunction_publication_assessment
Return the one assessment the card's verdict and its explanation share.
src/skillevaluator/reporting/benchmark.py:1171
↓ 3 callersFunction_publication_identity_key
Normalize canonical aliases without folding filesystem distinctions.
src/skillevaluator/reporting/base.py:2122
↓ 3 callersFunction_publication_safe_label
Sanitize a classified display label for a public benchmark card.
src/skillevaluator/reporting/benchmark.py:2209
↓ 3 callersFunction_publication_target_identity
(value: object)
src/skillevaluator/reporting/base.py:794
↓ 3 callersFunction_published_execution_errors
Return a redacted, byte-bounded, de-duplicated diagnostic sample.
src/skillevaluator/tier3/harbor/runner.py:729
↓ 3 callersFunction_python_pin
(text: str, *, extra: str = "", runner: bool = False)
src/skillevaluator/plugin_dependency_audit.py:131
↓ 3 callersFunction_quote_uri_path
Percent-encode a POSIX-style path for use in a URI. A file name that is not valid UTF-8 arrives with surrogate escapes; its original bytes ar
src/skillevaluator/reporting/sarif_reporter.py:104
↓ 3 callersMethod_raise_primary_with_cleanup
( self, primary_error: BaseException, cleanup_error: RuntimeError, secret_valu
src/skillevaluator/tier3/harbor/local_environment.py:1873
↓ 3 callersMethod_raw_container_states
( self, identities: Iterable[str], *, service: str, )
src/skillevaluator/tier3/harbor/secure_docker_environment.py:1621
↓ 3 callersFunction_raw_trial_rewards
Return compact raw Harbor reward dicts (internal + verbose keys stripped).
src/skillevaluator/evaluation/tier3_report.py:1405
↓ 3 callersMethod_read_bytes
Bounded, anchored, no-follow read counted against a read budget; raises :class:`SecurePathError`. ``config`` charges the read to the separate
src/skillevaluator/plugin_paths.py:163
↓ 3 callersFunction_read_call
(observation: str)
tests/tier3/test_harbor_template_secret_patterns.py:65
↓ 3 callersFunction_read_call
(observation: str)
tests/tier3/test_checks_secret_patterns.py:240
↓ 3 callersFunction_read_json
(path: Path)
src/skillevaluator/reporting/dashboard_data.py:558
↓ 3 callersMethod_read_json
(self)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:248
↓ 3 callersFunction_read_key
(path: Path)
src/skillevaluator/tier3/output_provenance.py:362
↓ 3 callersFunction_read_manifest
(skill_path: Path)
src/skillevaluator/harness/overlay.py:228
↓ 3 callersFunction_read_regular
(path: Path)
src/skillevaluator/tier3/plugin_prepare.py:776
↓ 3 callersFunction_read_regular
(path: Path)
src/skillevaluator/tier3/loose_parts.py:1322
↓ 3 callersFunction_read_retained
(path: Path, default)
ci/publish_reports.py:454
↓ 3 callersFunction_read_stable_bytes
Read one regular artifact through the pinned run root.
src/skillevaluator/tier3/results_location.py:177
↓ 3 callersFunction_reasons
(agent: dict)
tests/reporting/test_report_data_bounds.py:46
↓ 3 callersFunction_rebase
(result: ValidationResult, old: Path, new: Path)
src/skillevaluator/tier1/plugin_checks.py:83
↓ 3 callersFunction_rebase_custom_dockerfile_content
Return custom Dockerfile content layered on top of the eval base image. The base image already contains: python:3.12-slim, the system packages in
src/skillevaluator/tier3/harbor/adapter.py:3960
↓ 3 callersFunction_record_judge_call
Record the model that answered for the required judge in progress (see _call_required_judge).
src/skillevaluator/tier3/harbor/templates/eval.py:2300
↓ 3 callersFunction_record_skillspector_reference_summary
Count the referenced artifacts behind *result*'s AE1 evidence in its metadata. The counts come from the findings (:func:`~skillevaluator.skil
src/skillevaluator/validators/security.py:1093
↓ 3 callersMethod_redacted_cleanup_error
Return a fresh cleanup error with no reference to the raw exception.
src/skillevaluator/tier3/harbor/local_environment.py:1864
↓ 3 callersFunction_redacted_servers
Runnable servers with command/args/url redacted, exactly as the wrapper TOML does.
src/skillevaluator/tier3/plugin_native.py:1338
↓ 3 callersFunction_ref_covers
Whether every call ``specific`` matches is also matched by ``general`` (same type, glob over the text).
src/skillevaluator/tier3/eval_core/plugin_signals.py:1202
↓ 3 callersFunction_ref_label
(values: Sequence[str])
src/skillevaluator/tier3/eval_core/plugin_signals.py:3500
↓ 3 callersFunction_registry_vector_dimension
Return the width for work-budget checks without retaining normalized copies.
src/skillevaluator/embedding/registry.py:841
↓ 3 callersFunction_relative_path
(value: object)
src/skillevaluator/tier3/decoys.py:121
↓ 3 callersFunction_reliability
(markup)
tests/reporting/test_report_readability.py:21
↓ 3 callersFunction_remaining_deadline
(deadline: float)
src/skillevaluator/model_catalog.py:107
↓ 3 callersFunction_remove_tree_at
( parent_descriptor: int, name: str, *, expected_identity: tuple[int, int] | None = None, )
src/skillevaluator/tier3/harbor/secure_copy.py:916
↓ 3 callersFunction_render
(agents)
tests/reporting/test_diagnostic_trial_comparisons.py:33
↓ 3 callersFunction_render
(results)
tests/reporting/test_report_version_placement.py:36
↓ 3 callersFunction_render
(panel)
tests/harness/test_overlay.py:249
↓ 3 callersFunction_render_reports
Write ``result.json`` and render the canonical payload, ``tier3-benchmark.md`` and ``report.html``.
tests/tier3/test_baseline_timeouts.py:743
↓ 3 callersFunction_rendered_live_lines
(reporter: Any, *, width: int = 80)
tests/test_tier3_progress.py:293
↓ 3 callersMethod_reorganize_by_skill
Reorganize validation results by skill instead of by validator. Returns a dict like: { "skill-name": { "p
src/skillevaluator/reporting/html.py:2434
↓ 3 callersFunction_repeated_attempts_report
(trial_ids: tuple[str, ...])
tests/reporting/test_dashboard_data.py:422
↓ 3 callersFunction_report_formats_explicit
True when the user passed ``-r``/``--report`` on the command line. Catalog mode re-invokes ``validate`` through a child context that records
src/skillevaluator/cli.py:726
↓ 3 callersMethod_request
(self, method: str, path: str, payload=None)
ci/github_api.py:80
↓ 3 callersFunction_require
(condition: bool, message: str)
ci/smoke_image.py:147
↓ 3 callersFunction_require_object
(payload: Any)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1443
↓ 3 callersFunction_require_windows_artifact_resources
Fail before mutation when a filesystem cannot retain headroom.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:1118
↓ 3 callersMethod_requirement_lines
Join backslash continuations into ``(first physical line, text)`` pairs. Each backslash before a newline becomes a space, and the text is spl
src/skillevaluator/validators/dependencies.py:520
↓ 3 callersMethod_resize
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 3 callersMethod_resolve_bridge_api_key
()
src/skillevaluator/tier3/harbor/local_agents.py:725
↓ 3 callersFunction_resolve_harbor_result_entry_id
Resolve (entry_id, is_authoritative) from a Harbor result.json payload.
src/skillevaluator/tier3/harbor/collector.py:4173
↓ 3 callersFunction_resolve_path
``path`` normalized and, when relative and the directory is known, made absolute.
src/skillevaluator/tier3/eval_core/plugin_signals.py:2523
↓ 3 callersMethod_resolve_task_mcp_servers
Resolve rich MCP declarations from task mcp_servers.json if available.
src/skillevaluator/tier3/harbor/local_agents.py:360
↓ 3 callersFunction_resolve_vertex_model_id
Normalize user-supplied Claude model string to Vertex Anthropic model ID.
src/skillevaluator/tier3/harbor/runtime_preflight.py:1494
↓ 3 callersFunction_resolve_vertex_region
Resolve the Google Cloud region for Vertex AI from provider config or environment variables.
src/skillevaluator/tier3/harbor/runtime_preflight.py:1482
↓ 3 callersMethod_resolved_model_name
Let the native Vertex client select its endpoint after model parsing.
src/skillevaluator/tier3/harbor/local_agents.py:214
↓ 3 callersFunction_resolved_workdir
Return the directory *target* names from *workdir*, or None when the shell decides it at run time. None means the directory is unknown: *target*
src/skillevaluator/tier3/eval_core/checks.py:1369
↓ 3 callersMethod_restore_sidecar_service
( self, service: str, *, snapshot: _RawServiceSnapshot, )
src/skillevaluator/tier3/harbor/secure_docker_environment.py:1819
↓ 3 callersFunction_result_has_publication_target_conflict
(result: ValidationResult)
src/skillevaluator/reporting/base.py:875
↓ 3 callersMethod_results_to_dict
(self, results: list[ValidationResult])
src/skillevaluator/reporting/html.py:3034
↓ 3 callersFunction_reward
(case: str, *, has_skill: bool = True, **overrides: Any)
tests/tier3/test_skill_metric_applicability.py:269
↓ 3 callersFunction_reward_security
(reward: Mapping[str, Any])
src/skillevaluator/tier3/eval_core/runtime_evidence.py:235
↓ 3 callersFunction_rewrite_score
(path: Path, score: float)
tests/reporting/test_dashboard_app.py:94
↓ 3 callersMethod_rewrite_uploaded_script
(self, target: Path)
src/skillevaluator/tier3/harbor/local_environment.py:2024
↓ 3 callersFunction_rm_cd_workdirs
Return where ``cd target`` leaves a shell that is in one of *workdirs*. The result is added to *seen*; once that holds more than ``_RM_MAX_WO
src/skillevaluator/tier3/eval_core/checks.py:6318
↓ 3 callersFunction_rm_symbolic_link_command
Recognize ln/cp symbolic-link options without reading option values.
src/skillevaluator/tier3/eval_core/checks.py:5855
↓ 3 callersFunction_rm_token_workdirs
Follow cd, pushd and popd through shell *tokens* in order. Returns the directories each token's command starts in, and those the shell is lef
src/skillevaluator/tier3/eval_core/checks.py:6330
↓ 3 callersFunction_root_variable_plugin
(root: Path, manifest: str, command: str, record: str = _RECORD, launcher: str = _LAUNCHER)
tests/validators/test_plugin_static_risk_review_fixes.py:476
↓ 3 callersFunction_row
(report: dict)
tests/validators/test_skillspector_hook_surface.py:316
↓ 3 callersFunction_rubric_result
(payload: str)
tests/reporting/test_rich_markup_escaping.py:121
↓ 3 callersFunction_rules
Yield (media preludes, selector, declarations) for every style rule.
tests/reporting/test_report_redesign.py:414
↓ 3 callersFunction_run_doctor
Validate every option before any probe, then check each selected tier in order.
src/skillevaluator/cli.py:4361
↓ 3 callersFunction_run_generated_test_sh
Execute the generated test.sh script with the specified environment variables.
tests/tier3/test_judge_failure_artifacts.py:1564
↓ 3 callersMethod_run_pip_audit_on_file
Audit only explicit pins, without installing packages or running build code. Parse the untrusted file ourselves so pip cannot follow includes
src/skillevaluator/validators/dependencies.py:407
↓ 3 callersFunction_run_private
(tmp_path, monkeypatch, *, missing=None, publish=None, comparisons=True)
tests/tier3/test_private_runtime_report_acceptance.py:171
↓ 3 callersFunction_run_script
(name: str, *args: str, env: dict[str, str] | None = None)
tests/test_reference_skill_scripts.py:43
↓ 3 callersFunction_run_setup
(bundle: Path, root: Path, env: dict[str, str])
tests/tier3/test_private_loose_source.py:501
← previousnext →2,901–3,000 of 24,741, ranked by callers