MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 2 callersFunctionreap
()
src/skillevaluator/tier3/harbor/secure_docker_environment.py:744
↓ 2 callersMethodrecord_failure
Keep observer failures advisory and redact diagnostics before publication.
src/skillevaluator/tier3/plugin_evidence.py:582
↓ 2 callersFunctionrecord_problem
(directory: Path, message: str)
src/skillevaluator/utils/helpers.py:68
↓ 2 callersFunctionrecord_unverified_dependencies
Add the Tier 1 findings for refs the missing-dependency gate could not check, and for over-limit sections. Each identity problem gives one MEDIUM
src/skillevaluator/plugin_dependencies.py:928
↓ 2 callersMethodrecords
(self, key: str, raw: object)
src/skillevaluator/reporting/plugin_runtime.py:86
↓ 2 callersFunctionredact_bridge_text
Redact common credential-bearing text before bridge diagnostics are written.
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1431
↓ 2 callersFunctionref_name
Return the trailing resource name of a dependency ref, or ``None``.
src/skillevaluator/plugin_dependencies.py:301
↓ 2 callersFunctionrefusal_units
A refused line after complete commands, inside a compound, and before an invocation, at the top level and in ``-c`` payloads.
scripts/script_invocation_differential.py:803
↓ 2 callersFunctionrefuse_cancellation
()
tests/tier3/test_judge_failure_artifacts.py:653
↓ 2 callersFunctionrefuse_or_fall_back
The decision for an agent that cannot load the plugin natively, for ``reason``. ``native`` refuses: it raises :class:`PluginLoadError`. ``auto``
src/skillevaluator/tier3/plugin_native.py:2194
↓ 2 callersMethodrelease_request
Drop all server bookkeeping after a request socket is no longer owned.
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:935
↓ 2 callersFunctionremote_preflight
(repo: Path, name: str, branch: str, full: bool)
src/skillevaluator/ci_onboarding.py:503
↓ 2 callersFunctionremove_generated_output_root_if_owned
Remove an authenticated failed reservation without following replacements. Descriptor-capable platforms use identity-bound recursive removal. Oth
src/skillevaluator/tier3/output_provenance.py:759
↓ 2 callersFunctionrender
(wizard, repo, release, platform, branch, roots, options)
src/skillevaluator/_ci_onboarding_all_tiers.py:402
↓ 2 callersFunctionrender
(index)
tests/reporting/test_retained_html.py:273
↓ 2 callersMethodrender_all
(self, results: list[ValidationResult])
src/skillevaluator/reporting/benchmark.py:189
↓ 2 callersMethodrender_all
(self, results: list[ValidationResult])
src/skillevaluator/reporting/junit_reporter.py:450
↓ 2 callersFunctionrender_declarative_coverage
(console: Console, agents: object, *, text: Callable[[str], str] = str)
src/skillevaluator/reporting/declarative_coverage.py:66
↓ 2 callersFunctionrender_fixtures
Write full.html, stress.html and absent.html into ``directory`` and return it.
tests/reporting/browser_navigation_fixtures.py:115
↓ 2 callersFunctionrender_plugin_runtime_reports
Compose static and runtime evidence without collecting or changing results. A saved report follows the same path as a live report. The private ad
src/skillevaluator/reporting/plugin_runtime_render.py:59
↓ 2 callersFunctionrender_plugin_static_reports
Render a private static bundle through existing reporters with one reduction. This function is deliberately absent from CLI dispatch and the publ
src/skillevaluator/reporting/plugin_static_render.py:139
↓ 2 callersFunctionrender_setup_script
Render ``/skilleval/native/setup.sh``: stage per-run config, then write the load census. The script is POSIX sh, never exits non-zero (a failed s
src/skillevaluator/tier3/plugin_native.py:886
↓ 2 callersFunctionreport_dir
Name the skill report directory behind the private scanner directory the runner passes.
tests/ci/test_run_tier1.py:79
↓ 2 callersFunctionreport_value
A value from plugin config (a command argument, a package spec) for reports. It is withheld whole as ``<value withheld>`` when it is shaped like
src/skillevaluator/validators/url_policy.py:824
↓ 2 callersFunctionrepository_git
(workspace, *arguments)
tests/ci/test_run_tier1.py:721
↓ 2 callersFunctionrequire_skill
(path: Path)
src/skillevaluator/tier_workflows.py:25
↓ 2 callersFunctionrequired_credentials
Credential variable names required by the selected tier, never values.
ci/model_config.py:74
↓ 2 callersMethodreset_finding_counts
Zero the error/warning totals and per-severity counters.
src/skillevaluator/models/result.py:326
↓ 2 callersFunctionresolve
(value: str)
tests/reporting/test_report_redesign.py:567
↓ 2 callersMethodresolveDataElementOptions
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersMethodresolveDatasetElementOptions
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersFunctionresolve_benchmark_policy
Resolve required publication tiers from persisted policy metadata. Each policy key is resolved independently. The selected canonical payload
src/skillevaluator/reporting/base.py:1872
↓ 2 callersFunctionresolve_explicit_or_latest_results
Resolve a specific run path or the latest run for refinement/reporting.
src/skillevaluator/tier3/results_location.py:1123
↓ 2 callersFunctionresolve_plugin_path
Convert a plugin manifest file path to its plugin root directory.
src/skillevaluator/cli_core.py:141
↓ 2 callersFunctionresolve_repo_context_root
Return the exact source root used by Tier 3 repo-context staging.
src/skillevaluator/utils/path_security.py:107
↓ 2 callersFunctionresolve_skill_path
Convert SKILL.md file path to its parent directory.
src/skillevaluator/cli_core.py:124
↓ 2 callersFunctionresolve_workflows_path
Convert workflow-rules.mdc path to its parent directory.
src/skillevaluator/cli_core.py:134
↓ 2 callersFunctionresult_publication_evidence_dict
Return a fresh canonical producer marker safe for report serialization.
src/skillevaluator/publication_evidence.py:102
↓ 2 callersFunctionresult_publication_target_conflict_marker
Return a safe conflict marker when a producer persisted that field.
src/skillevaluator/reporting/base.py:867
↓ 2 callersFunctionresult_publication_target_dict
Return a fresh, canonical three-field result target safe for output.
src/skillevaluator/reporting/base.py:846
↓ 2 callersFunctionretain_diagnostics
Retain whitelisted worker diagnostics below ``destination/diagnostics``. The caller supplies an evaluator-owned output root. The optional command
ci/diagnostics.py:191
↓ 2 callersFunctionretained_run
Resolve one retained run without following run-directory aliases.
src/skillevaluator/tier3/version_comparison.py:19
↓ 2 callersMethodreviews
(self, branch)
src/skillevaluator/_ci_onboarding_all_tiers.py:290
↓ 2 callersFunctionro
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersFunctionroute_logs_to
Print log records through *target* (a live view's console) until called with ``None``.
src/skillevaluator/logging_config.py:27
↓ 2 callersFunctionrt
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 2 callersMethodrun
(self, instruction, environment, context)
tests/test_harbor_timeout_recovery_docker.py:41
↓ 2 callersFunctionrun_context_optimization_check
( skill_path: Path, *, threshold: float = 0.80, model: str | None = None, llm_model: str |
src/skillevaluator/tier2/commands.py:79
↓ 2 callersFunctionrun_dedup_scan
( skill_path: Path, *, threshold: float = 0.80, llm_model: str | None = None, model: str |
src/skillevaluator/tier2/commands.py:97
↓ 2 callersFunctionrun_harbor_eval
Run Tier 3 with protected, coordinator-owned progress lifecycle.
src/skillevaluator/tier3/harbor/runner.py:7064
↓ 2 callersFunctionrun_lint_scripts
(target_path: Path)
src/skillevaluator/tier1/commands.py:619
↓ 2 callersFunctionrun_nemo_gym_eval
Run the selected arms on NeMo Gym and return the shared results mapping.
src/skillevaluator/tier3/nemo_gym/runner.py:830
↓ 2 callersFunctionrun_publish_loop
Run the real publish step shell with stand-ins for git and the publisher.
tests/ci/test_publisher_workflow.py:282
↓ 2 callersMethodrunning
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 2 callersMethodruns_any
Whether a file one of ``facts`` runs is a downloaded file. Files match by normalized path, or by base name when either side names the
src/skillevaluator/plugin_component_risk.py:1955
↓ 2 callersFunctionruns_schema_check
True when a run with these ``--checks``/``--skip`` values runs the schema check.
src/skillevaluator/tier1/commands.py:265
↓ 2 callersFunctionruns_secrets_scan
True when a run with these ``--checks``/``--skip`` values executes the Gitleaks scan.
src/skillevaluator/tier1/commands.py:270
↓ 2 callersMethodruntime
Stay on Docker: the hardened Compose runner and containment use the Docker CLI.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:1336
↓ 2 callersFunctionruntime_bin_dirs
Return managed bin directories for only the requested local agents.
src/skillevaluator/tier3/harbor/local_runtime.py:101
↓ 2 callersFunctionruntime_command_roots
Return exact executable/dependency reads and proven package roots.
src/skillevaluator/tier3/harbor/local_runtime.py:470
↓ 2 callersFunctionruntime_report_from_payload
Validate a bounded, detached snapshot supplied by an in-process caller.
src/skillevaluator/reporting/plugin_runtime_io.py:133
↓ 2 callersFunctionsafe
(value)
src/skillevaluator/tier3/history.py:494
↓ 2 callersFunctionsafe_relative
(value: str)
src/skillevaluator/_ci_onboarding_git.py:126
↓ 2 callersMethodsame_identity
Whether both snapshots describe the same file: one volume and one file index.
src/skillevaluator/utils/secure_fs.py:108
↓ 2 callersMethodsame_line
Reject a line break before the next token: `=>` and a postfix `++` may not follow one.
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:908
↓ 2 callersFunctionsanitize_reference_caveat_metadata
Retain only bounded, canonical caveat metadata at report boundaries.
src/skillevaluator/skillspector_reference_caveats.py:216
↓ 2 callersFunctionsanitized_outputs
Read and sanitize bounded outputs before writing any shared artifact. The evaluator records the target as the CI passed it, an absolute path in
ci/secret_redaction.py:108
↓ 2 callersMethodsave_cache
Deprecated compatibility alias for :meth:`save_catalog`.
src/skillevaluator/embedding/registry.py:709
↓ 2 callersFunctionsave_case_findings
Replace the advisory cache atomically, including an existing symlink itself.
src/skillevaluator/tier3/case_summary.py:901
↓ 2 callersFunctionsave_files
(repo: Path, files: dict[str, str], expected: dict[str, str] | None = None)
src/skillevaluator/ci_onboarding.py:299
↓ 2 callersFunctionsaved_feedback_for_run
Return the report feedback saved in ``result.json`` when it belongs to this run.
src/skillevaluator/evaluation/tier3_feedback_chat.py:192
↓ 2 callersFunctionscan_outputs
(reports)
tests/ci/test_run_tier1.py:1272
↓ 2 callersFunctionscan_text
Return every denied rule match in decoded text.
scripts/check_oss_boundary.py:598
↓ 2 callersFunctionscore
(module, command)
tests/tier3/test_cache_cleanup_scoring.py:19
↓ 2 callersMethodscore
Preflight the complete workload, normalize each catalog vector once, and batch targets.
src/skillevaluator/deduplication/plugin/catalog.py:322
↓ 2 callersMethodsection
( self, key: str, title: str, columns: tuple[str, ...], rows: list[lis
src/skillevaluator/reporting/plugin_static.py:196
↓ 2 callersFunctionsecurity_code_literals
``text`` plus its string literals joined two ways, so ``["~", ".ssh"].join("/")`` reads as a path.
src/skillevaluator/tier3/eval_core/plugin_security.py:3128
↓ 2 callersFunctionsecurity_entries
The entries of ``entries`` that ``path`` names or lies inside (a glob can name several).
src/skillevaluator/tier3/eval_core/plugin_security.py:2463
↓ 2 callersFunctionsecurity_secret_kind
``("secret", pattern)`` when ``text`` holds a key, ``("harness", pattern)`` when every key in it is the harness's own model or cloud credential, e
src/skillevaluator/tier3/eval_core/plugin_security.py:3161
↓ 2 callersFunctionsecurity_tree_entries
Entries under the directory ``path`` (a home, ``/``, or a parent of a store) that a full read reaches.
src/skillevaluator/tier3/eval_core/plugin_security.py:2470
↓ 2 callersFunctionselect_changed_skills
Return existing changed skills and a bounded, deterministic audit record. Commits must be full object IDs and the working tree must match ``head`
ci/select_changed_skills.py:200
↓ 2 callersMethodselected_files
Require each alias target to be an independently discovered regular file; return the selection.
src/skillevaluator/utils/secure_fs.py:1063
↓ 2 callersFunctionself_plugin_entries
(profile, catalog)
src/skillevaluator/deduplication/plugin/catalog_checks.py:51
↓ 2 callersMethodsend_signal
(self, signum: int)
tests/reporting/test_dashboard_cli.py:39
↓ 2 callersMethodsetCenterPoint
(t,e,i,s)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersMethodset_secret_values
(self, values: list[str] | tuple[str, ...] | set[str])
src/skillevaluator/tier3/harbor/progress.py:141
↓ 2 callersMethodset_secret_values
(self, values: list[str] | tuple[str, ...] | set[str])
src/skillevaluator/tier3/harbor/progress.py:241
↓ 2 callersMethodset_secret_values
(self, values: list[str] | tuple[str, ...] | set[str])
src/skillevaluator/tier3/harbor/runner.py:5344
↓ 2 callersFunctionsetup_changed
Return whether any changed path (both sides of a rename) is CI setup.
ci/select_changed_skills.py:40
↓ 2 callersFunctionshared_case_scores
Score two arms per case on the dimensions both arms scored for that case. An arm without the skill under test has no Discoverability or Efficienc
src/skillevaluator/tier3/harbor/stats.py:426
↓ 2 callersFunctionshared_not_applicable_reason
Return the distinct reasons *rewards* record for their N/A *metric*, joined, or ``None``.
src/skillevaluator/tier3/harbor/metrics.py:406
↓ 2 callersMethodsize
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersFunctionskill_efficiency_not_applicable_reason
Return why an eval case gives ``skill_efficiency`` nothing to measure, or ``None`` when it applies.
src/skillevaluator/tier3/eval_core/checks.py:8441
↓ 2 callersFunctionskill_metrics_applicable
Return whether an arm can be scored on ``skill_execution`` and ``skill_efficiency``. The arm that carries the skill or plugin always can. An arm
src/skillevaluator/tier3/harbor/templates/eval.py:4082
↓ 2 callersFunctionskill_root
Validate one workspace-relative skill root and return its normalized form. The Tier 1 runner and changed-skill selection share these rules. A roo
ci/select_changed_skills.py:68
↓ 2 callersFunctionskill_trees
Walk validated relative roots and return the skills found under them.
ci/run_tier1.py:113
↓ 2 callersFunctionskillspector_reasoning_effort
Return the SkillSpector reasoning effort SkillEvaluator sets for ``model``. Only GPT-6 models get one. Claude models behind OpenAI-compatible gat
src/skillevaluator/provider_config.py:206
↓ 2 callersFunctionskipped
(name)
tests/reporting/test_tier3_validate_cases.py:652
↓ 2 callersFunctionsnapshot
(target: Path, handle)
src/skillevaluator/ci_onboarding.py:341
↓ 2 callersFunctionsource_validation_full_report
Name the report file of this run that lists every failed Tier 3 source check. ``report_formats`` are the formats the run actually emits. HTML Ful
src/skillevaluator/reporting/base.py:2613
← previousnext →5,201–5,300 of 24,741, ranked by callers