MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 5 callersFunctionfind_unscanned_plugin_skill_manifests
Plugin-root-relative ``SKILL.md`` paths that bundled-skill discovery skips, and whether the search finished. These sit in a scan-excluded folder
src/skillevaluator/plugin_member_paths.py:125
↓ 5 callersFunctionft
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 5 callersFunctiong
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 5 callersFunctiongenerate
(parsed, **_kwargs)
tests/tier3/test_dataset_baseline_fairness.py:1501
↓ 5 callersMethodgetPointPosition
(t,e,i=0)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callersFunctionget_adc_token_expiry
Return the recorded UTC epoch expiry for an ADC token, if known.
src/skillevaluator/provider_config.py:283
↓ 5 callersFunctionget_final_response
Get the last text-bearing agent message from the trajectory.
src/skillevaluator/tier3/eval_core/atif_helpers.py:138
↓ 5 callersMethodget_install_hint
(self)
tests/validators/test_private_claude_parity.py:29
↓ 5 callersFunctiongit
(*args)
tests/ci/test_run_tier1_image.py:47
↓ 5 callersFunctiongrade
(trajectory)
tests/tier3/test_plugin_evidence_review.py:47
↓ 5 callersFunctiongym_launch_environment
Return Gym's process environment: the host variables a Harbor backend may receive, plus its credentials. Relative host paths (Docker TLS, CA bund
src/skillevaluator/tier3/nemo_gym/runner.py:756
↓ 5 callersFunctionh
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 5 callersFunctionharbor_timeout_job_recovered
Check one failed single-trial launch against its successful fresh attempt.
src/skillevaluator/tier3/harbor/collector.py:6830
↓ 5 callersMethodhas
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 5 callersFunctionin_memory_reports
(reports, manifest)
tests/ci/test_benchmark_reports.py:272
↓ 5 callersMethodindependent_warnings
Return warnings no structured finding repeats, such as why a scanner did not complete.
src/skillevaluator/models/result.py:657
↓ 5 callersFunctioninteger
(env: dict, key: str, default: int, minimum: int, maximum: int)
ci/run_tiers.py:77
↓ 5 callersFunctioninvoke
(api, output)
tests/ci/test_prepare_comments.py:57
↓ 5 callersMethodisHorizontal
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callersFunctionis_claude_5_5_or_later
Return whether ``model`` is a Claude 5.5 or later model.
src/skillevaluator/provider_config.py:172
↓ 5 callersFunctionis_env_reference
True when *value* is one environment reference that carries no literal value of its own. ``$VAR``, ``${VAR}``, ``${VAR:-}`` (an empty default), a
src/skillevaluator/validators/url_policy.py:235
↓ 5 callersFunctionis_frontmatter_hook_source
Whether a hooks source is a skill's or command's frontmatter ``hooks`` block (named ``<file>#hooks``). Such hooks run only while their skill or c
src/skillevaluator/tier3/plugin_native.py:302
↓ 5 callersFunctionkept_with_failed_baseline
Return whether ``agent`` stays scored although its baseline condition failed. Without tolerance a failed arm always fails its agent, so this shap
src/skillevaluator/tier3/harbor/baseline_timeouts.py:308
↓ 5 callersFunctionlive_case_marks
Attempt marks of an in-flight run, read from its Harbor job directories or Gym rollouts. Returns ``(marks, running)``: every staged case appears
src/skillevaluator/tier3/case_summary.py:1662
↓ 5 callersFunctionload_dataset_snapshot
Validate the recorded schema, then normalize in memory without rewriting it.
src/skillevaluator/tier3/harbor/report_data.py:203
↓ 5 callersFunctionload_quality_config_with_provenance
Load the config and describe it. Only the file name is kept, so reports do not leak local paths. The digest covers the rules the file itself defi
src/skillevaluator/quality_config.py:311
↓ 5 callersFunctionloose_parts_task_staging
The runner's seam: the loose arm's native task staging for ``agent``. Returns the :class:`~skillevaluator.tier3.harbor.native_staging.NativeTaskS
src/skillevaluator/tier3/loose_parts.py:609
↓ 5 callersFunctionlstat_walk
Inspect ``root / relative`` one component at a time with ``lstat``, never following a link. The walk stops at the first component that is missing
src/skillevaluator/utils/secure_fs.py:170
↓ 5 callersFunctionmake_view_console
A console for view-level output. Resolves ``sys.stdout`` at CALL time: do not call inside the Tier 3 engine-capture window (``redirect_stdout
src/skillevaluator/reporting/console_ui.py:243
↓ 5 callersFunctionmanaged_install_argv
The npm command that installs ``agent``'s pinned CLI into the managed runtime root.
src/skillevaluator/tier3/harbor/local_runtime.py:522
↓ 5 callersFunctionmatcher_scope
Classify a tool matcher: ``all`` (every tool), ``bash`` (matches Bash), or ``narrow``. Follows Claude Code's matcher rules without running the pl
src/skillevaluator/plugin_component_risk.py:1531
↓ 5 callersFunctionmissing_tier1_publication_checks
Return applicable built-in Tier 1 checks that no executed result covers. Built-in Tier 1 producers record the content type's whole lineup, so a r
src/skillevaluator/reporting/base.py:2051
↓ 5 callersFunctionnative_refusal
The first bypass refusal for a component type ``adapter`` stages natively, or ``None``. A component that enables a permission bypass blocks only
src/skillevaluator/tier3/plugin_native.py:236
↓ 5 callersFunctionne
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callersFunctionnemo_gym_bin
Locate the ``gym`` CLI, preferring the pinned worker named by ``SKILL_EVAL_NEMO_GYM_BIN``.
src/skillevaluator/tier3/nemo_gym/backend.py:57
↓ 5 callersFunctionnemo_gym_capability_errors
Reject configurations the NeMo Gym backend cannot run equivalently, before anything starts.
src/skillevaluator/tier3/nemo_gym/backend.py:189
↓ 5 callersFunctionnemo_gym_preflight_errors
Return every option, config and setting error the NeMo Gym backend would reject, before any dataset work.
src/skillevaluator/tier3/workflow.py:261
↓ 5 callersFunctionnormalized_trial_timing
Validate one trial's ``timing``; invalid or untagged values stay unavailable, not zero.
src/skillevaluator/tier3/harbor/resource_usage.py:24
↓ 5 callersFunctionpaired_case_bootstrap
Expanded percentile bootstrap CI for the mean paired per-case lift. ``treatment`` and ``control`` map case ids to case-level mean scores. Only
src/skillevaluator/tier3/harbor/stats.py:581
↓ 5 callersFunctionparse_env_grant
Parse ``NAME``, ``NAME=HOST`` or ``NAME@SERVER``; raise ``ValueError`` when malformed.
src/skillevaluator/tier3/mcp_proof.py:147
↓ 5 callersFunctionparse_environment_kwargs
Parse repeatable ``--ek key=value`` CLI pairs via ``parse_environment_kwarg_overrides``.
src/skillevaluator/tier3/commands.py:415
↓ 5 callersFunctionparse_retry_after
Parse a Retry-After header as seconds or HTTP date, falling back to default.
src/skillevaluator/inference/retry.py:55
↓ 5 callersFunctionpartial_result
()
tests/validators/test_skillspector_diagnostics.py:24
↓ 5 callersFunctionpermission_flag_issues
Find agent-CLI permission flags in any config/command string or argv list, in one bounded walk. Flags and options that disable approvals or the s
src/skillevaluator/validators/mcp_static.py:3310
↓ 5 callersMethodplugin_name
(self, source: NativePluginSource)
src/skillevaluator/tier3/plugin_native.py:1379
↓ 5 callersMethodprint_summary
Print only the summary table (no failure details, no overall verdict). Used for progressive/interim CLI output -- e.g. flushing Tier 1 and
src/skillevaluator/reporting/cli.py:185
↓ 5 callersFunctionprivate_agent_import_path
Classify known setup failures without replacing an unsupported backend's agent. Unknown wrappers retain their own setup semantics and are not eli
src/skillevaluator/tier3/plugin_native.py:2349
↓ 5 callersFunctionpublication_source_entry_is_excluded
Return whether one existing source entry is omitted by the v2 recipe. Callers traversing a tree must invoke this for each entry before descending
src/skillevaluator/publication_identity.py:158
↓ 5 callersMethodread
(self, _size: int)
tests/test_secure_docker_environment.py:145
↓ 5 callersFunctionread_bounded
Read at most ``max_bytes`` bytes from an open descriptor, 64 KiB at a time. With ``truncate`` the first ``max_bytes`` bytes are returned and the
src/skillevaluator/utils/secure_fs.py:119
↓ 5 callersFunctionread_rollouts
Read a Gym rollouts JSONL file, skipping blank lines. A line Gym was still writing when it was stopped (timeout, OOM, full disk) is skipped a
src/skillevaluator/tier3/nemo_gym/results.py:30
↓ 5 callersMethodrecord_scanner
Record that scanner *name* ran, so reports show which scanner versions produced the result.
src/skillevaluator/models/result.py:713
↓ 5 callersMethodredirect_request
(self, _request, _file, _code, _message, _headers, _url)
ci/github_api.py:26
↓ 5 callersFunctionreference_summary_message
The one-line text of a valid *summary*, the same in every report, else ``None``. It says only that each finding names its file and any reasons, b
src/skillevaluator/skillspector_references.py:475
↓ 5 callersFunctionreference_summary_record
*summary* with only its count fields when they are valid, else ``None``.
src/skillevaluator/skillspector_references.py:446
↓ 5 callersMethodrefresh
Resolve the executable again, e.g. after a managed install.
src/skillevaluator/utils/tool_runner.py:180
↓ 5 callersMethodremaining
(self)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:475
↓ 5 callersFunctionremote
(repo: Path, platform: str = "auto")
src/skillevaluator/_ci_onboarding_all_tiers.py:134
↓ 5 callersMethodremoveEventListener
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callersMethodrender
(self, result: ValidationResult)
src/skillevaluator/reporting/html.py:3448
↓ 5 callersMethodrender_all
(self, results, *, plugin_static=None, plugin_runtime=None)
tests/reporting/test_plugin_evidence_report.py:140
↓ 5 callersFunctionrender_comment_summary
Return one compact Markdown table; never read paths or mutate evidence. Callers derive each skill's ``metrics`` from verified retained JSON using
ci/comment_summary.py:112
↓ 5 callersFunctionrenderers
()
tests/reporting/test_private_runtime_render.py:40
↓ 5 callersFunctionreporters
()
tests/reporting/test_private_runtime_render.py:44
↓ 5 callersFunctionrepository
(tmp_path)
tests/ci/test_prepare_security_baselines.py:18
↓ 5 callersFunctionresolve_evaluated_source
Merge the two orchestration channels into one identity. Fields are resolved individually rather than whole-dict, so a mistyped field in one c
src/skillevaluator/source_identity.py:232
↓ 5 callersFunctionresolve_results_root
Resolve the per-skill results root for writes. Precedence: 1. command ``--results-dir`` root 2. ``SKILLEVALUATOR_RESULTS_DIR`` root 3
src/skillevaluator/tier3/results_location.py:78
↓ 5 callersFunctionresult_has_execution_evidence
Return whether a non-skipped validator proves that it executed work.
src/skillevaluator/reporting/base.py:2029
↓ 5 callersMethodroute
(t,e,i,s)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callersMethodrow
(self, *values: object, archive: bool = False)
src/skillevaluator/reporting/native_evidence.py:121
↓ 5 callersFunctionrows_csv
Export visible quantitative columns with spreadsheet-safe text cells.
src/skillevaluator/reporting/dashboard_app.py:124
↓ 5 callersFunctionrun
(workspace, **extra)
tests/ci/test_prepare_security_baselines.py:37
↓ 5 callersFunctionsafe_child
Return a validated strict child of *base*, rejecting symlink escapes.
src/skillevaluator/tier3/case_ids.py:141
↓ 5 callersFunctionsafe_llm_model
Return a model identifier fit for reports, or ``unresolved``.
src/skillevaluator/inference/diagnostics.py:46
↓ 5 callersFunctionsanitize_skillspector_provenance_metadata
Project only the canonical, report-safe SkillSpector provenance fields of result metadata.
src/skillevaluator/skillspector_provenance.py:426
↓ 5 callersFunctionselect_prior_version
Check a source before execution; latest excludes the current skill revision.
src/skillevaluator/tier3/version_comparison.py:97
↓ 5 callersFunctionskill_roots
Parse SKILL_PATHS into 1 to 100 unique, sorted and validated roots.
ci/select_changed_skills.py:93
↓ 5 callersFunctionskipped_checks
Resolve ``--skip`` names (aliases included) to canonical check names.
src/skillevaluator/tier1/commands.py:260
↓ 5 callersMethodsleep
(self, seconds: float)
tests/tier3/test_judge_failure_artifacts.py:324
↓ 5 callersFunctionstamp_publication_target
Atomically stamp newly generated results with their source identity. Every preexisting claim is checked before any result is changed. A claim
src/skillevaluator/publication_identity.py:799
↓ 5 callersMethodstart
(self, plan)
tests/test_nemo_gym_backend.py:1963
↓ 5 callersFunctionstatic_result
()
tests/reporting/test_report_navigation.py:54
↓ 5 callersFunctionstill_running
Wait up to WAIT for the processes to end; return those still running.
tests/ci/test_run_tier1.py:1519
↓ 5 callersMethodstop_service
(self, service: str)
src/skillevaluator/tier3/harbor/secure_docker_environment.py:2379
↓ 5 callersFunctionsummarize_messages
Count string messages that no structured finding represents. Findings also add a mirrored string to a result's ``errors`` or ``warnings`` lis
src/skillevaluator/reporting/finding_summary.py:286
↓ 5 callersFunctionsummarize_tier3
Return (ran, passed, rows, skip_reason) for the agent-eval result. Failed Tier 3 source checks add a count row. ``report_formats`` are the fo
src/skillevaluator/reporting/console_ui.py:1575
↓ 5 callersMethodtext
(self, value: object)
src/skillevaluator/reporting/native_evidence.py:104
↓ 5 callersFunctiontoken_usage_markdown_lines
Return a Markdown token usage section: agent rows, then case rows in a collapsed table. ``agent_label`` gives the display name of an agent (``Non
src/skillevaluator/reporting/tier3_token_usage.py:259
↓ 5 callersFunctionunwrap_launch_command
Strip launch wrappers so the runner they start is classified. ``cmd /c <command line>``, ``powershell -Command <line>``, ``env`` (with its op
src/skillevaluator/validators/mcp_static.py:1726
↓ 5 callersFunctionvalid_archive
()
tests/ci/test_publish_reports.py:166
↓ 5 callersFunctionvalidate
Validate a skill through all three tiers, with automatic dataset preparation. Runs Tier 1 static, security, and quality checks (which gate the ex
src/skillevaluator/cli.py:2524
↓ 5 callersFunctionvalidate_declarative_checks
(value: object)
src/skillevaluator/tier3/eval_core/declarative_checks.py:34
↓ 5 callersFunctionvalidate_manifest
Return a copied canonical manifest; reject contradictory gate claims. ``expected_tiers`` defaults to all three tiers. A single-tier producer can
ci/tier_manifest.py:75
↓ 5 callersFunctionvalidate_max_entries
(max_entries: int)
src/skillevaluator/embedding/limits.py:9
↓ 5 callersFunctionvalidate_max_scalar_comparisons
(max_scalar_comparisons: int)
src/skillevaluator/embedding/limits.py:14
↓ 5 callersFunctionvalidate_pydantic_model
Validate data against a Pydantic model, adding errors to result. Args: model_class: The Pydantic model class to validate against
src/skillevaluator/validators/frontmatter_parser.py:95
↓ 5 callersFunctionvalidate_report_scope
(scope: str, results: list)
src/skillevaluator/reporting/tier3_scope.py:11
↓ 5 callersFunctionvalidate_runtime_root
Return a canonical dedicated runtime root without exposing host home.
src/skillevaluator/tier3/harbor/local_runtime.py:79
↓ 5 callersFunctionve
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callersMethodwith_identity_permission
(self)
src/skillevaluator/_ci_onboarding_workflows.py:83
← previousnext →1,901–2,000 of 24,741, ranked by callers