MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 1 callersFunctionrun
(command, **kwargs)
tests/test_doctor.py:60
↓ 1 callersFunctionrun
(label: str)
tests/test_harbor_local_mode.py:2306
↓ 1 callersFunctionrun
Run the same checks as CI, retaining screenshots and JUnit results.
tests/reporting/browser_navigation_checks.py:13
↓ 1 callersFunctionrun
(name, path, **kwargs)
tests/cli/test_validate_defaults.py:43
↓ 1 callersFunctionrun
(version: str)
tests/harness/test_overlay.py:1178
↓ 1 callersMethodrun
Account for the numbers, literals, colons, and commas in ``raw[start:end]``.
src/skillevaluator/utils/structured_data.py:320
↓ 1 callersMethodrun
(self, instruction, environment: BaseEnvironment, context)
src/skillevaluator/tier3/harbor/local_agents.py:950
↓ 1 callersFunctionrun_admitted
(task_index: int, task_name: str, attempt: int, admissions: list[dict[str, Any]])
src/skillevaluator/tier3/harbor/runner.py:4857
↓ 1 callersFunctionrun_agent_runtime_preflight
Start one real agent task and stop before the full A/B matrix.
src/skillevaluator/tier3/harbor/runtime_preflight.py:1975
↓ 1 callersFunctionrun_cancelled
()
tests/test_secure_docker_environment.py:248
↓ 1 callersFunctionrun_cancelled
()
tests/test_harbor_local_mode.py:3252
↓ 1 callersFunctionrun_cancelled_during_create
()
tests/test_harbor_local_mode.py:3758
↓ 1 callersFunctionrun_case
(args: argparse.Namespace, size: int, layout: str, mode: str, malformed: int)
scripts/benchmark_tier2_scale.py:110
↓ 1 callersFunctionrun_check
Run one offline check against a single skill directory. Never writes reports. *quality_config* replaces the packaged quality-default.yaml for the
src/skillevaluator/precommit.py:210
↓ 1 callersFunctionrun_cleanup
()
tests/test_harbor_secure_docker_environment.py:7541
↓ 1 callersFunctionrun_command
Bound subprocess lifetime and stop the entire process group on timeout.
ci/run_tiers.py:176
↓ 1 callersMethodrun_config
``run_config["loose_parts"]``: per agent, whether its loose arm runs and why not.
src/skillevaluator/tier3/harbor/loose_parts_arm.py:113
↓ 1 callersFunctionrun_feedback_chat
Chat about one completed Tier 3 run directory; nothing is written to disk. Returns 0 when the chat ends normally and 1 when the run or provider c
src/skillevaluator/tier3/commands.py:1395
↓ 1 callersFunctionrun_harness_check
Evaluate each skill harness by harness, updating a live panel as each finishes.
src/skillevaluator/harness/report.py:244
↓ 1 callersFunctionrun_marker
(run_id: int, attempt: int)
ci/comment_report.py:64
↓ 1 callersFunctionrun_models_command
List models visible to the canonically selected public provider.
src/skillevaluator/model_commands.py:18
↓ 1 callersFunctionrun_pairs
(tmp_path, budget, cancellation=None, **kwargs)
tests/tier3/test_budget_reservation.py:292
↓ 1 callersFunctionrun_pii_scan
( target_path: Path, *, llm_verify: bool = False, pii_waivers: Path | None = None, )
src/skillevaluator/tier1/commands.py:601
↓ 1 callersFunctionrun_plugin_skill_context_dedup
Run C-intra for unique effective skill directories with an aggregate call allowance.
src/skillevaluator/deduplication/plugin/commands.py:82
↓ 1 callersFunctionrun_private_evaluation
Run the existing engine for a prepared plugin, then publish evidence, then clean up. Evidence is published before cleanup removes the operation-o
src/skillevaluator/tier3/plugin_execution.py:343
↓ 1 callersFunctionrun_probe
()
tests/test_harbor_local_mode.py:1853
↓ 1 callersFunctionrun_repeated_cancellation
()
tests/test_harbor_local_mode.py:4038
↓ 1 callersFunctionrun_rubric_eval
(target_path: Path, *, min_score: int = QUALITY_DEFAULT_MIN_SCORE)
src/skillevaluator/tier1/commands.py:569
↓ 1 callersFunctionrun_scan
(argv, *, abort, **options)
tests/ci/test_run_tier1.py:1747
↓ 1 callersFunctionrun_security_scan
( target_path: Path, *, use_llm: bool = False, llm_verify: bool = False, skillspector_base
src/skillevaluator/tier1/commands.py:582
↓ 1 callersFunctionrun_smoke
Require repeatable identities, findings, membership and coverage.
scripts/ci/smoke_openclaw_plugins.py:136
↓ 1 callersFunctionrun_tier1_workflow
Validate local configuration before optional scanner preparation and checks.
src/skillevaluator/tier_workflows.py:67
↓ 1 callersFunctionrun_tier2_workflow
( skill_path: Path, *, catalog: Path | None, threshold: float, similarity_threshold: float
src/skillevaluator/tier_workflows.py:133
↓ 1 callersFunctionrun_timeout
()
tests/test_harbor_secure_docker_environment.py:7483
↓ 1 callersFunctionrun_timeout
()
tests/test_harbor_local_mode.py:2828
↓ 1 callersFunctionrun_unit
(task_index: int, task_name: str, attempt: int)
src/skillevaluator/tier3/harbor/runner.py:4844
↓ 1 callersFunctionrun_verifier_main
Score a small trajectory the way test.sh runs eval.py, without judge credentials.
scripts/ci/smoke_harbor_verifier.py:80
↓ 1 callersFunctionrun_with_callback
()
tests/test_harbor_secure_docker_environment.py:8123
↓ 1 callersFunctionrun_with_key
(name: str, *extra: str)
tests/test_commands.py:693
↓ 1 callersMethodruns
(self, position: int)
src/skillevaluator/plugin_component_risk.py:3029
↓ 1 callersFunctionruns_security_scan
True when a run with these ``--checks``/``--skip`` values executes the SkillSpector scan.
src/skillevaluator/tier1/commands.py:275
↓ 1 callersFunctionruntime_report_for_results
Read a validated display snapshot without opening a source or run directory.
src/skillevaluator/reporting/plugin_reports.py:61
↓ 1 callersFunctionsa
(t,e,i,s)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunctionsafe_eval
Evaluate arithmetic expression using AST — no eval().
src/skillevaluator/tier3/reference_skills/calculator/scripts/calc.py:24
↓ 1 callersFunctionsalvage
(text: str)
src/skillevaluator/tier3/eval_core/llm_judge.py:1097
↓ 1 callersFunctionsalvage
(text)
src/skillevaluator/tier3/harbor/templates/eval.py:3814
↓ 1 callersFunctionsanitize
(item)
src/skillevaluator/tier3/harbor/templates/eval.py:1199
↓ 1 callersFunctionsanitize_input_schema
An MCP server's tool ``inputSchema`` cut to the subset argument checks run, or ``None``. Keeps ``type``, ``properties``, ``required``, ``enum``,
src/skillevaluator/tier3/eval_core/plugin_signals.py:997
↓ 1 callersFunctionsatisfies_gate
(record)
ci/tier_manifest.py:193
↓ 1 callersMethodsaturate
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunctionsave
(target: Path, path: Path)
tests/tier3/test_decoys.py:424
↓ 1 callersFunctionsave_plugin_runtime_report
Atomically retain one bounded sanitized snapshot in the selected run directory.
src/skillevaluator/reporting/plugin_runtime_io.py:146
↓ 1 callersFunctionscan
()
tests/ci/test_run_tier1.py:1560
↓ 1 callersMethodscan
()
src/skillevaluator/validators/security.py:3202
↓ 1 callersFunctionscan_archive
Scan a wheel or source distribution in memory without extracting it.
scripts/check_oss_boundary.py:1025
↓ 1 callersFunctionscan_file
( path: Path, *, require_source_provenance: bool = False, expected_source: ExpectedSource | No
scripts/ci/check_public_benchmarks.py:765
↓ 1 callersFunctionscan_repository
Scan the tracked public release surface, or the equivalent tree without Git metadata.
scripts/check_oss_boundary.py:899
↓ 1 callersFunctionscan_workers
Read the bound on concurrent skill scans; 1 scans one skill at a time.
ci/run_tier1.py:177
↓ 1 callersMethodscoped_exec_env
(self, _env: dict[str, str])
tests/tier3/test_harbor_local_agents.py:50
↓ 1 callersFunctionscored
(metric)
src/skillevaluator/tier3/harbor/templates/eval.py:4135
↓ 1 callersFunctionsection
(key, title, value)
tests/reporting/test_report_browser.py:273
↓ 1 callersFunctionsecurity_agent_anchors
``[(directory, "~" form)]`` for the agent's home and config directories, longest first.
src/skillevaluator/tier3/eval_core/plugin_security.py:2371
↓ 1 callersFunctionsecurity_code_reads
Path pieces of interpreter code or of an undecodable wrapper, as normalized paths.
src/skillevaluator/tier3/eval_core/plugin_security.py:3138
↓ 1 callersFunctionsecurity_default_cwd
The agent's starting directory: the workspace the environment names, else ``/workspace``.
src/skillevaluator/tier3/eval_core/plugin_security.py:2401
↓ 1 callersFunctionsecurity_destructive_label
The first destructive form in a command: a forced ``rm`` outside ``/tmp`` (``shell_rm`` from the shell words, or in interpreter code and quoted co
src/skillevaluator/tier3/eval_core/plugin_security.py:3300
↓ 1 callersFunctionsecurity_patch_paths
Normalized paths an apply_patch call (tool or shell form) writes: every Add, Update, Delete and Move-to header, resolved against the call's ``work
src/skillevaluator/tier3/eval_core/plugin_security.py:3316
↓ 1 callersFunctionsecurity_scan
Trace-level security scan using agent-issued actions as primary evidence. ``user_messages`` are the user steps; harness-written ones (skill bodie
src/skillevaluator/tier3/eval_core/plugin_security.py:3406
↓ 1 callersFunctionsecurity_shell_scan
Read one shell command: ``{"reads", "writes", "destructive", "phrases", "env_dump", "executed", "cwd"}``. ``reads`` are ``(path, tree)`` pairs: e
src/skillevaluator/tier3/eval_core/plugin_security.py:3059
↓ 1 callersFunctionsecurity_write_entry
The protected entry a written ``path`` is, else the credential store it lies in, else ``None``.
src/skillevaluator/tier3/eval_core/plugin_security.py:2480
↓ 1 callersMethodselect
(self, timeout: float | None = None)
tests/test_harbor_local_mode.py:193
↓ 1 callersMethodselect
(self, timeout=None)
tests/tier3/test_judge_failure_artifacts.py:639
↓ 1 callersFunctionselect_attempts
Place each tier's newest artifact from this or an earlier attempt of the run. GitHub's "Re-run failed jobs" reruns only failed jobs and their dep
ci/run_tiers.py:635
↓ 1 callersFunctionselect_baseline
Select fresh evidence, or return bounded rejection diagnostics for latest.
src/skillevaluator/tier3/baseline_store.py:362
↓ 1 callersFunctionselect_roots
(roots: list[dict], explicit: tuple[str, ...], no_input: bool)
src/skillevaluator/ci_onboarding.py:810
↓ 1 callersFunctionselect_skills
(workspace: Path, env: dict)
ci/run_tiers.py:99
↓ 1 callersFunctionsend_unstubbed_request
()
tests/tier3/test_ragas_judge_evidence.py:828
↓ 1 callersFunctionserve
Run a bridge bound to loopback only.
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:811
↓ 1 callersMethodsetDatasetVisibility
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodsetDimensions
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodsetDimensions
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodset_harness
Show the advisory harness panel after the Tier 1 block and *note* in the verdict.
src/skillevaluator/reporting/console_ui.py:647
↓ 1 callersMethodset_secret_values
(self, values: list[str] | tuple[str, ...] | set[str])
src/skillevaluator/tier3/harbor/progress.py:915
↓ 1 callersMethodset_secret_values
(self, _values)
tests/test_tier3_progress.py:1071
↓ 1 callersMethodsettings
(self)
src/skillevaluator/plugin_discovery.py:1474
↓ 1 callersFunctionsettle
(value: str)
src/skillevaluator/reporting/plugin_runtime_io.py:48
↓ 1 callersFunctionsettle
Keep a tolerated agent scored on its with-skill arm. The agent's status becomes ``succeeded`` with no agent-level errors, and ``record`` is a
src/skillevaluator/tier3/harbor/baseline_timeouts.py:254
↓ 1 callersMethodsetup
(self, environment)
tests/test_harbor_timeout_recovery_docker.py:38
↓ 1 callersFunctionsetup_logging
Setup root logging configuration. Args: verbose: If True, set logging level to DEBUG
src/skillevaluator/logging_config.py:88
↓ 1 callersFunctionsetup_only_failure
The setup failure that makes ``result`` safe to rerun, or ``None``. Every expected trial in every executed arm must have a complete, typed se
src/skillevaluator/tier3/plugin_execution.py:299
↓ 1 callersFunctionshell_program
The program string when ``tokens`` run a shell interpreter with ``-c`` (``bash -c '<program>'``).
src/skillevaluator/validators/mcp_static.py:1896
↓ 1 callersMethodshow
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunctionshow_next_steps
(name: str, full: bool, setup_url: str | None = None, *, repo: Path | None = None)
src/skillevaluator/ci_onboarding.py:96
↓ 1 callersFunctionshow_test_pr
(name: str, url: str, *, existing: bool = False, full: bool = True)
src/skillevaluator/ci_onboarding.py:114
↓ 1 callersFunctionsiblings
(i: int, j: int)
src/skillevaluator/deduplication/intra_skill/semantic_clustering.py:106
↓ 1 callersFunctionsignal_owned
(sig: int)
src/skillevaluator/tier3/nemo_gym/runner.py:535
↓ 1 callersFunctionskill_changed_since_run
Whether the skill's current content differs from the snapshot the run evaluated. Returns ``False`` when either digest is unavailable.
src/skillevaluator/evaluation/tier3_feedback_chat.py:166
↓ 1 callersFunctionskill_command
(tier: int, skill: Path, output: Path, env: dict, *, catalog: Path | None = None)
ci/run_tiers.py:199
↓ 1 callersFunctionskill_execution_not_applicable_reason
Return why an eval case gives ``skill_execution`` nothing to score, or ``None`` when it applies. A negative case still scores: it checks that the
src/skillevaluator/tier3/eval_core/checks.py:8429
↓ 1 callersFunctionskill_path
(path: str)
src/skillevaluator/tier3/eval_core/checks.py:7291
↓ 1 callersFunctionskill_results_name
Return the directory name used under an external results root.
src/skillevaluator/tier3/results_location.py:59
← previousnext →9,101–9,200 of 24,741, ranked by callers