Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/NVIDIA/SkillEvaluator
/ functions
Functions
24,741 in github.com/NVIDIA/SkillEvaluator
⨍
Functions
24,741
◇
Types & classes
1,292
↳
Endpoints
703
↓ 5 callers
Function
find_unscanned_plugin_skill_manifests
Plugin-root-relative ``SKILL.md`` paths that bundled-skill discovery skips, and whether the search finished. These sit in a scan-excluded folder
src/skillevaluator/plugin_member_paths.py:125
↓ 5 callers
Function
ft
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 5 callers
Function
g
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 5 callers
Function
generate
(parsed, **_kwargs)
tests/tier3/test_dataset_baseline_fairness.py:1501
↓ 5 callers
Method
getPointPosition
(t,e,i=0)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callers
Function
get_adc_token_expiry
Return the recorded UTC epoch expiry for an ADC token, if known.
src/skillevaluator/provider_config.py:283
↓ 5 callers
Function
get_final_response
Get the last text-bearing agent message from the trajectory.
src/skillevaluator/tier3/eval_core/atif_helpers.py:138
↓ 5 callers
Method
get_install_hint
(self)
tests/validators/test_private_claude_parity.py:29
↓ 5 callers
Function
git
(*args)
tests/ci/test_run_tier1_image.py:47
↓ 5 callers
Function
grade
(trajectory)
tests/tier3/test_plugin_evidence_review.py:47
↓ 5 callers
Function
gym_launch_environment
Return Gym's process environment: the host variables a Harbor backend may receive, plus its credentials. Relative host paths (Docker TLS, CA bund
src/skillevaluator/tier3/nemo_gym/runner.py:756
↓ 5 callers
Function
h
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 5 callers
Function
harbor_timeout_job_recovered
Check one failed single-trial launch against its successful fresh attempt.
src/skillevaluator/tier3/harbor/collector.py:6830
↓ 5 callers
Method
has
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 5 callers
Function
in_memory_reports
(reports, manifest)
tests/ci/test_benchmark_reports.py:272
↓ 5 callers
Method
independent_warnings
Return warnings no structured finding repeats, such as why a scanner did not complete.
src/skillevaluator/models/result.py:657
↓ 5 callers
Function
integer
(env: dict, key: str, default: int, minimum: int, maximum: int)
ci/run_tiers.py:77
↓ 5 callers
Function
invoke
(api, output)
tests/ci/test_prepare_comments.py:57
↓ 5 callers
Method
isHorizontal
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callers
Function
is_claude_5_5_or_later
Return whether ``model`` is a Claude 5.5 or later model.
src/skillevaluator/provider_config.py:172
↓ 5 callers
Function
is_env_reference
True when *value* is one environment reference that carries no literal value of its own. ``$VAR``, ``${VAR}``, ``${VAR:-}`` (an empty default), a
src/skillevaluator/validators/url_policy.py:235
↓ 5 callers
Function
is_frontmatter_hook_source
Whether a hooks source is a skill's or command's frontmatter ``hooks`` block (named ``<file>#hooks``). Such hooks run only while their skill or c
src/skillevaluator/tier3/plugin_native.py:302
↓ 5 callers
Function
kept_with_failed_baseline
Return whether ``agent`` stays scored although its baseline condition failed. Without tolerance a failed arm always fails its agent, so this shap
src/skillevaluator/tier3/harbor/baseline_timeouts.py:308
↓ 5 callers
Function
live_case_marks
Attempt marks of an in-flight run, read from its Harbor job directories or Gym rollouts. Returns ``(marks, running)``: every staged case appears
src/skillevaluator/tier3/case_summary.py:1662
↓ 5 callers
Function
load_dataset_snapshot
Validate the recorded schema, then normalize in memory without rewriting it.
src/skillevaluator/tier3/harbor/report_data.py:203
↓ 5 callers
Function
load_quality_config_with_provenance
Load the config and describe it. Only the file name is kept, so reports do not leak local paths. The digest covers the rules the file itself defi
src/skillevaluator/quality_config.py:311
↓ 5 callers
Function
loose_parts_task_staging
The runner's seam: the loose arm's native task staging for ``agent``. Returns the :class:`~skillevaluator.tier3.harbor.native_staging.NativeTaskS
src/skillevaluator/tier3/loose_parts.py:609
↓ 5 callers
Function
lstat_walk
Inspect ``root / relative`` one component at a time with ``lstat``, never following a link. The walk stops at the first component that is missing
src/skillevaluator/utils/secure_fs.py:170
↓ 5 callers
Function
make_view_console
A console for view-level output. Resolves ``sys.stdout`` at CALL time: do not call inside the Tier 3 engine-capture window (``redirect_stdout
src/skillevaluator/reporting/console_ui.py:243
↓ 5 callers
Function
managed_install_argv
The npm command that installs ``agent``'s pinned CLI into the managed runtime root.
src/skillevaluator/tier3/harbor/local_runtime.py:522
↓ 5 callers
Function
matcher_scope
Classify a tool matcher: ``all`` (every tool), ``bash`` (matches Bash), or ``narrow``. Follows Claude Code's matcher rules without running the pl
src/skillevaluator/plugin_component_risk.py:1531
↓ 5 callers
Function
missing_tier1_publication_checks
Return applicable built-in Tier 1 checks that no executed result covers. Built-in Tier 1 producers record the content type's whole lineup, so a r
src/skillevaluator/reporting/base.py:2051
↓ 5 callers
Function
native_refusal
The first bypass refusal for a component type ``adapter`` stages natively, or ``None``. A component that enables a permission bypass blocks only
src/skillevaluator/tier3/plugin_native.py:236
↓ 5 callers
Function
ne
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callers
Function
nemo_gym_bin
Locate the ``gym`` CLI, preferring the pinned worker named by ``SKILL_EVAL_NEMO_GYM_BIN``.
src/skillevaluator/tier3/nemo_gym/backend.py:57
↓ 5 callers
Function
nemo_gym_capability_errors
Reject configurations the NeMo Gym backend cannot run equivalently, before anything starts.
src/skillevaluator/tier3/nemo_gym/backend.py:189
↓ 5 callers
Function
nemo_gym_preflight_errors
Return every option, config and setting error the NeMo Gym backend would reject, before any dataset work.
src/skillevaluator/tier3/workflow.py:261
↓ 5 callers
Function
normalized_trial_timing
Validate one trial's ``timing``; invalid or untagged values stay unavailable, not zero.
src/skillevaluator/tier3/harbor/resource_usage.py:24
↓ 5 callers
Function
paired_case_bootstrap
Expanded percentile bootstrap CI for the mean paired per-case lift. ``treatment`` and ``control`` map case ids to case-level mean scores. Only
src/skillevaluator/tier3/harbor/stats.py:581
↓ 5 callers
Function
parse_env_grant
Parse ``NAME``, ``NAME=HOST`` or ``NAME@SERVER``; raise ``ValueError`` when malformed.
src/skillevaluator/tier3/mcp_proof.py:147
↓ 5 callers
Function
parse_environment_kwargs
Parse repeatable ``--ek key=value`` CLI pairs via ``parse_environment_kwarg_overrides``.
src/skillevaluator/tier3/commands.py:415
↓ 5 callers
Function
parse_retry_after
Parse a Retry-After header as seconds or HTTP date, falling back to default.
src/skillevaluator/inference/retry.py:55
↓ 5 callers
Function
partial_result
()
tests/validators/test_skillspector_diagnostics.py:24
↓ 5 callers
Function
permission_flag_issues
Find agent-CLI permission flags in any config/command string or argv list, in one bounded walk. Flags and options that disable approvals or the s
src/skillevaluator/validators/mcp_static.py:3310
↓ 5 callers
Method
plugin_name
(self, source: NativePluginSource)
src/skillevaluator/tier3/plugin_native.py:1379
↓ 5 callers
Method
print_summary
Print only the summary table (no failure details, no overall verdict). Used for progressive/interim CLI output -- e.g. flushing Tier 1 and
src/skillevaluator/reporting/cli.py:185
↓ 5 callers
Function
private_agent_import_path
Classify known setup failures without replacing an unsupported backend's agent. Unknown wrappers retain their own setup semantics and are not eli
src/skillevaluator/tier3/plugin_native.py:2349
↓ 5 callers
Function
publication_source_entry_is_excluded
Return whether one existing source entry is omitted by the v2 recipe. Callers traversing a tree must invoke this for each entry before descending
src/skillevaluator/publication_identity.py:158
↓ 5 callers
Method
read
(self, _size: int)
tests/test_secure_docker_environment.py:145
↓ 5 callers
Function
read_bounded
Read at most ``max_bytes`` bytes from an open descriptor, 64 KiB at a time. With ``truncate`` the first ``max_bytes`` bytes are returned and the
src/skillevaluator/utils/secure_fs.py:119
↓ 5 callers
Function
read_rollouts
Read a Gym rollouts JSONL file, skipping blank lines. A line Gym was still writing when it was stopped (timeout, OOM, full disk) is skipped a
src/skillevaluator/tier3/nemo_gym/results.py:30
↓ 5 callers
Method
record_scanner
Record that scanner *name* ran, so reports show which scanner versions produced the result.
src/skillevaluator/models/result.py:713
↓ 5 callers
Method
redirect_request
(self, _request, _file, _code, _message, _headers, _url)
ci/github_api.py:26
↓ 5 callers
Function
reference_summary_message
The one-line text of a valid *summary*, the same in every report, else ``None``. It says only that each finding names its file and any reasons, b
src/skillevaluator/skillspector_references.py:475
↓ 5 callers
Function
reference_summary_record
*summary* with only its count fields when they are valid, else ``None``.
src/skillevaluator/skillspector_references.py:446
↓ 5 callers
Method
refresh
Resolve the executable again, e.g. after a managed install.
src/skillevaluator/utils/tool_runner.py:180
↓ 5 callers
Method
remaining
(self)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:475
↓ 5 callers
Function
remote
(repo: Path, platform: str = "auto")
src/skillevaluator/_ci_onboarding_all_tiers.py:134
↓ 5 callers
Method
removeEventListener
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callers
Method
render
(self, result: ValidationResult)
src/skillevaluator/reporting/html.py:3448
↓ 5 callers
Method
render_all
(self, results, *, plugin_static=None, plugin_runtime=None)
tests/reporting/test_plugin_evidence_report.py:140
↓ 5 callers
Function
render_comment_summary
Return one compact Markdown table; never read paths or mutate evidence. Callers derive each skill's ``metrics`` from verified retained JSON using
ci/comment_summary.py:112
↓ 5 callers
Function
renderers
()
tests/reporting/test_private_runtime_render.py:40
↓ 5 callers
Function
reporters
()
tests/reporting/test_private_runtime_render.py:44
↓ 5 callers
Function
repository
(tmp_path)
tests/ci/test_prepare_security_baselines.py:18
↓ 5 callers
Function
resolve_evaluated_source
Merge the two orchestration channels into one identity. Fields are resolved individually rather than whole-dict, so a mistyped field in one c
src/skillevaluator/source_identity.py:232
↓ 5 callers
Function
resolve_results_root
Resolve the per-skill results root for writes. Precedence: 1. command ``--results-dir`` root 2. ``SKILLEVALUATOR_RESULTS_DIR`` root 3
src/skillevaluator/tier3/results_location.py:78
↓ 5 callers
Function
result_has_execution_evidence
Return whether a non-skipped validator proves that it executed work.
src/skillevaluator/reporting/base.py:2029
↓ 5 callers
Method
route
(t,e,i,s)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callers
Method
row
(self, *values: object, archive: bool = False)
src/skillevaluator/reporting/native_evidence.py:121
↓ 5 callers
Function
rows_csv
Export visible quantitative columns with spreadsheet-safe text cells.
src/skillevaluator/reporting/dashboard_app.py:124
↓ 5 callers
Function
run
(workspace, **extra)
tests/ci/test_prepare_security_baselines.py:37
↓ 5 callers
Function
safe_child
Return a validated strict child of *base*, rejecting symlink escapes.
src/skillevaluator/tier3/case_ids.py:141
↓ 5 callers
Function
safe_llm_model
Return a model identifier fit for reports, or ``unresolved``.
src/skillevaluator/inference/diagnostics.py:46
↓ 5 callers
Function
sanitize_skillspector_provenance_metadata
Project only the canonical, report-safe SkillSpector provenance fields of result metadata.
src/skillevaluator/skillspector_provenance.py:426
↓ 5 callers
Function
select_prior_version
Check a source before execution; latest excludes the current skill revision.
src/skillevaluator/tier3/version_comparison.py:97
↓ 5 callers
Function
skill_roots
Parse SKILL_PATHS into 1 to 100 unique, sorted and validated roots.
ci/select_changed_skills.py:93
↓ 5 callers
Function
skipped_checks
Resolve ``--skip`` names (aliases included) to canonical check names.
src/skillevaluator/tier1/commands.py:260
↓ 5 callers
Method
sleep
(self, seconds: float)
tests/tier3/test_judge_failure_artifacts.py:324
↓ 5 callers
Function
stamp_publication_target
Atomically stamp newly generated results with their source identity. Every preexisting claim is checked before any result is changed. A claim
src/skillevaluator/publication_identity.py:799
↓ 5 callers
Method
start
(self, plan)
tests/test_nemo_gym_backend.py:1963
↓ 5 callers
Function
static_result
()
tests/reporting/test_report_navigation.py:54
↓ 5 callers
Function
still_running
Wait up to WAIT for the processes to end; return those still running.
tests/ci/test_run_tier1.py:1519
↓ 5 callers
Method
stop_service
(self, service: str)
src/skillevaluator/tier3/harbor/secure_docker_environment.py:2379
↓ 5 callers
Function
summarize_messages
Count string messages that no structured finding represents. Findings also add a mirrored string to a result's ``errors`` or ``warnings`` lis
src/skillevaluator/reporting/finding_summary.py:286
↓ 5 callers
Function
summarize_tier3
Return (ran, passed, rows, skip_reason) for the agent-eval result. Failed Tier 3 source checks add a count row. ``report_formats`` are the fo
src/skillevaluator/reporting/console_ui.py:1575
↓ 5 callers
Method
text
(self, value: object)
src/skillevaluator/reporting/native_evidence.py:104
↓ 5 callers
Function
token_usage_markdown_lines
Return a Markdown token usage section: agent rows, then case rows in a collapsed table. ``agent_label`` gives the display name of an agent (``Non
src/skillevaluator/reporting/tier3_token_usage.py:259
↓ 5 callers
Function
unwrap_launch_command
Strip launch wrappers so the runner they start is classified. ``cmd /c <command line>``, ``powershell -Command <line>``, ``env`` (with its op
src/skillevaluator/validators/mcp_static.py:1726
↓ 5 callers
Function
valid_archive
()
tests/ci/test_publish_reports.py:166
↓ 5 callers
Function
validate
Validate a skill through all three tiers, with automatic dataset preparation. Runs Tier 1 static, security, and quality checks (which gate the ex
src/skillevaluator/cli.py:2524
↓ 5 callers
Function
validate_declarative_checks
(value: object)
src/skillevaluator/tier3/eval_core/declarative_checks.py:34
↓ 5 callers
Function
validate_manifest
Return a copied canonical manifest; reject contradictory gate claims. ``expected_tiers`` defaults to all three tiers. A single-tier producer can
ci/tier_manifest.py:75
↓ 5 callers
Function
validate_max_entries
(max_entries: int)
src/skillevaluator/embedding/limits.py:9
↓ 5 callers
Function
validate_max_scalar_comparisons
(max_scalar_comparisons: int)
src/skillevaluator/embedding/limits.py:14
↓ 5 callers
Function
validate_pydantic_model
Validate data against a Pydantic model, adding errors to result. Args: model_class: The Pydantic model class to validate against
src/skillevaluator/validators/frontmatter_parser.py:95
↓ 5 callers
Function
validate_report_scope
(scope: str, results: list)
src/skillevaluator/reporting/tier3_scope.py:11
↓ 5 callers
Function
validate_runtime_root
Return a canonical dedicated runtime root without exposing host home.
src/skillevaluator/tier3/harbor/local_runtime.py:79
↓ 5 callers
Function
ve
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callers
Method
with_identity_permission
(self)
src/skillevaluator/_ci_onboarding_workflows.py:83
← previous
next →
1,901–2,000 of 24,741, ranked by callers