MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 1 callersFunction_reward_from_harbor_result
(result: dict[str, Any])
src/skillevaluator/tier3/harbor/collector.py:3982
↓ 1 callersFunction_reward_has_failing_signal
(reward: dict[str, Any])
src/skillevaluator/tier3/harbor/report.py:1096
↓ 1 callersFunction_reward_identity_is_publishable
Validate the effective case identity before score aggregation.
src/skillevaluator/tier3/harbor/collector.py:4571
↓ 1 callersFunction_reward_overall
The score policy's mean of the five dimensions, leaving out judged metrics recorded as N/A. An N/A metric is left out of its dimension, and a dim
src/skillevaluator/tier3/harbor/templates/eval.py:4128
↓ 1 callersFunction_reward_trajectory_path
(trial_root: Path, step_name: str | None)
src/skillevaluator/tier3/harbor/collector.py:1696
↓ 1 callersFunction_reward_trial_context
Return ``(trial_root, trial_name, step_name)`` for a Harbor reward file. Harbor single-step tasks write ``<trial>/verifier/reward.json``. Native
src/skillevaluator/tier3/harbor/collector.py:1679
↓ 1 callersFunction_reward_with_custom_metric
()
tests/tier3/test_report_renders_refs.py:59
↓ 1 callersFunction_reward_with_dict_refs
Reward whose evidence_refs are the new dict shape.
tests/tier3/test_report_renders_refs.py:32
↓ 1 callersFunction_reward_with_normalized_tool_refs
()
tests/tier3/test_suggestion_grounding.py:377
↓ 1 callersFunction_reward_with_path_only_refs
()
tests/tier3/test_suggestion_grounding.py:410
↓ 1 callersMethod_rewrite_bridge_client_command
(self, command: str)
src/skillevaluator/tier3/harbor/local_agents.py:694
↓ 1 callersFunction_rewrite_path_prefix
Recursively rewrite ``old`` path prefixes in a parsed JSON structure.
src/skillevaluator/validators/security.py:1067
↓ 1 callersMethod_rewrite_uploaded_scripts
(self, target: Path)
src/skillevaluator/tier3/harbor/local_environment.py:2015
↓ 1 callersFunction_rich_skip_reason
Return one bounded, single-line skip reason inert to Rich markup.
src/skillevaluator/reporting/cli.py:107
↓ 1 callersFunction_rich_source_error
Return one failed Tier 3 source check as a bounded line inert to Rich markup.
src/skillevaluator/reporting/cli.py:90
↓ 1 callersFunction_rm_brace_sequence
Return the words of a bash ``{a..b[..step]}`` sequence, [] when *body* is not one, None past *limit*.
src/skillevaluator/tier3/eval_core/checks.py:6106
↓ 1 callersFunction_rm_brace_words
Expand bash brace lists and sequences in an rm target, or None past *limit* words or *budget*. ``${...}`` is a parameter expansion, not a brace l
src/skillevaluator/tier3/eval_core/checks.py:6133
↓ 1 callersFunction_rm_call_workdirs
Return the directories a call's command starts in. The call's ``workdir`` or ``cwd`` argument replaces *workdirs*, where the previous call le
src/skillevaluator/tier3/eval_core/checks.py:6301
↓ 1 callersFunction_rm_dot_glob_paths
Spell *word* with each combination of its dot globs before the last component read as "..". A pattern such as ``.*`` matches ".." in bash, so ``/
src/skillevaluator/tier3/eval_core/checks.py:6184
↓ 1 callersFunction_rm_find_exec_protected
True when a find whose -exec, -execdir, -ok or -okdir runs ``tokens[rm_index]`` starts at a protected path. find hands the command every file und
src/skillevaluator/tier3/eval_core/checks.py:6264
↓ 1 callersFunction_rm_group_end
Find a substitution's closing parenthesis without expanding its output.
src/skillevaluator/tier3/eval_core/checks.py:5406
↓ 1 callersFunction_rm_operand_follows_find
True when an rm target run by find -exec can be a file find found. That is the ``{}`` placeholder, an expansion that can carry it (``"$1"``),
src/skillevaluator/tier3/eval_core/checks.py:6255
↓ 1 callersFunction_rm_sequence_workdirs
Return the directories each command of a plain sequence starts in, in one walk so popd and cd - work.
src/skillevaluator/tier3/eval_core/checks.py:6387
↓ 1 callersFunction_rm_sequential_commands
Order only plain sequences; keep whole-action evidence for control flow.
src/skillevaluator/tier3/eval_core/checks.py:6472
↓ 1 callersFunction_rm_shell_workdirs
Return the directories the shell is in after *command_text*, starting from *workdirs*.
src/skillevaluator/tier3/eval_core/checks.py:6400
↓ 1 callersFunction_rm_stdin_code
Associate a single input document with the command that reads it.
src/skillevaluator/tier3/eval_core/checks.py:5738
↓ 1 callersFunction_rm_stdin_codes
Inspect each input document, including multiple documents on one line.
src/skillevaluator/tier3/eval_core/checks.py:5800
↓ 1 callersFunction_rm_symlink_call_seen
Inspect Python/Node source without treating strings or comments as calls.
src/skillevaluator/tier3/eval_core/checks.py:5887
↓ 1 callersFunction_rm_symlink_fallback
Keep direct link commands detectable when whole-action budgets expire.
src/skillevaluator/tier3/eval_core/checks.py:5923
↓ 1 callersFunction_rm_targets_fallback
Confine parse failures and budgets to candidate deletion commands. Raw separators can cut a quoted word or substitution short; that fragment
src/skillevaluator/tier3/eval_core/checks.py:6427
↓ 1 callersFunction_rm_tmp_operand_is_exempt
Preserve lexical /tmp cleanup unless the trace gives link evidence.
src/skillevaluator/tier3/eval_core/checks.py:6011
↓ 1 callersFunction_rm_xargs_context
True for an xargs child rm, False for its option values/help, else unknown.
src/skillevaluator/tier3/eval_core/checks.py:5594
↓ 1 callersMethod_root_aliases
Variables a hook command sets to the plugin root or a path under it, each with its plain placeholder. Reading ``$R/x.sh`` as a plugin script
src/skillevaluator/plugin_component_risk.py:3172
↓ 1 callersFunction_root_cause
(exc: BaseException)
src/skillevaluator/tier3/mcp_proof.py:553
↓ 1 callersFunction_root_default_chain
``(first variable, length)`` of the parameter-default chain that starts ``value`` when every alternative in it is one of ``names`` (``${CLAUDE_PLU
src/skillevaluator/plugin_component_risk.py:2482
↓ 1 callersMethod_root_directory
The root-relative directory a ``cd`` target names, or ``None`` when it is not under the plugin root or still names a variable.
src/skillevaluator/plugin_component_risk.py:3147
↓ 1 callersFunction_root_error_in_logs
Find the agent CLI's own error line in its log (failure.json truncates long messages).
src/skillevaluator/tier3/case_summary.py:1008
↓ 1 callersFunction_root_prefixes
The spellings of *scan_root*'s directory that an absolute report path may start with.
src/skillevaluator/skillspector_references.py:88
↓ 1 callersMethod_root_skill
A root ``SKILL.md`` makes a single-skill plugin (Cursor) when no skills are declared or bundled.
src/skillevaluator/plugin_discovery.py:729
↓ 1 callersFunction_root_variable_pattern
Match ``$NAME/``, ``${NAME}/``, ``$env:NAME/``, ``%NAME%/``, and an optional placeholder form.
src/skillevaluator/validators/skillspector_scope.py:887
↓ 1 callersMethod_rooted
A command word (quotes removed) with the command's root variables expanded and a root default chain made plain, so ``"$R/x.sh"`` after ``R="${
src/skillevaluator/plugin_component_risk.py:3164
↓ 1 callersFunction_routing
Mirror the grader: ``should_trigger`` wins, then a truthy ``expected_skill``; neither means unlabeled.
src/skillevaluator/tier3/next_steps.py:57
↓ 1 callersFunction_routing_ref
Whether a ref is about routing (check 15) rather than tool selection (check 22). Typed ``Skill:``/``Agent:``/``Command:`` refs are routing and ``
src/skillevaluator/tier3/eval_core/plugin_signals.py:3648
↓ 1 callersFunction_row
Build one row; ``case`` is ``None`` for an ``All cases`` sum, which alone carries a note.
src/skillevaluator/reporting/tier3_token_usage.py:124
↓ 1 callersFunction_row_key
(row: dict[str, Any])
src/skillevaluator/evaluation/tier3_feedback_chat.py:897
↓ 1 callersFunction_row_text
(row)
tests/test_artifact_applicability.py:70
↓ 1 callersFunction_rubric_report_problem
Return why a parsed judge reply cannot be scored against ``criteria``, or ``None`` when it can.
src/skillevaluator/validators/rubric_eval.py:218
↓ 1 callersFunction_rule_descriptor
(finding: Finding, validator_name: str)
src/skillevaluator/reporting/sarif_reporter.py:62
↓ 1 callersFunction_rule_flag
``alwaysApply`` as the native adapters read it: YAML ``true`` or the string ``"true"`` (any case).
src/skillevaluator/plugin_context_cost.py:209
↓ 1 callersFunction_rule_globs
Cursor ``globs`` or Claude ``paths``: a list, or a comma-separated string, of glob patterns.
src/skillevaluator/tier3/plugin_native.py:1142
↓ 1 callersFunction_rule_has_frontmatter
Whether a rule file opens with a closed ``---`` frontmatter block, as the native adapters test it.
src/skillevaluator/plugin_context_cost.py:214
↓ 1 callersFunction_rule_names
Everything a profile rule can name: skill facts, the model setup, ``always`` and ``roots`` (every root, with ``roots.project`` and ``roots.user``
src/skillevaluator/harness/overlay.py:890
↓ 1 callersFunction_rule_patterns
Cursor ``globs`` or Claude ``paths``: a list, or a comma-separated string, of glob patterns.
src/skillevaluator/plugin_context_cost.py:220
↓ 1 callersFunction_rule_source
The plugin-relative rule file, when the rule name is one (``rules/<name>``).
src/skillevaluator/tier3/loose_parts.py:1284
↓ 1 callersFunction_rule_view
(row: CostRow, harness: str, load_mode: str)
src/skillevaluator/plugin_context_cost.py:262
↓ 1 callersMethod_rules
Stage the rules with the plugin arm's own writer, so path and bytes are the same in both arms.
src/skillevaluator/tier3/loose_parts.py:893
↓ 1 callersFunction_run
(tool: Any, args: list[str], directory: Path, env: dict[str, str], timeout: int)
src/skillevaluator/plugin_dependency_audit.py:466
↓ 1 callersMethod_run
(self, root: Path, parity: dict[str, Any])
src/skillevaluator/validators/claude_plugin_validate.py:563
↓ 1 callersMethod_run
(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch, *args: str)
tests/test_benchmark_source_provenance.py:1465
↓ 1 callersFunction_run_agent_eval_or_skip
Run Tier 3 live agent evaluation and fold the result into the combined report. Returns an ``AGENT_EVAL`` :class:`ValidationResult` carrying the c
src/skillevaluator/cli.py:984
↓ 1 callersFunction_run_agent_pair
Run ordinary and privately requested arms under one per-agent trial budget.
src/skillevaluator/tier3/harbor/runner.py:5146
↓ 1 callersFunction_run_case
(task_name: str)
src/skillevaluator/tier3/harbor/runner.py:5017
↓ 1 callersFunction_run_coverage
(payload: dict, summary: dict, agents: dict)
src/skillevaluator/reporting/insights_view.py:28
↓ 1 callersFunction_run_custom_only
()
src/skillevaluator/tier3/harbor/templates/custom_grader_runner.py:377
↓ 1 callersFunction_run_dedup_or_skip
Run Tier 2 dedup when possible, else return a non-failing skipped result. Dedup is on by default for ``validate`` but needs the ``tier2`` extra a
src/skillevaluator/cli.py:743
↓ 1 callersFunction_run_default_plus_custom
()
src/skillevaluator/tier3/harbor/templates/custom_grader_runner.py:338
↓ 1 callersFunction_run_error
(trial_dir: Path, run_dir: Path)
src/skillevaluator/tier3/case_summary.py:1023
↓ 1 callersFunction_run_file_checks
Run spec-specific validation on an existing file.
src/skillevaluator/tier3/evals_spec.py:1150
↓ 1 callersFunction_run_harbor_eval_impl
Run a public Harbor evaluation with and without the target skill.
src/skillevaluator/tier3/harbor/runner.py:5442
↓ 1 callersFunction_run_hygiene
Resolve links against the full snapshot; audit each unit's own metadata.
src/skillevaluator/tier1/plugin_checks.py:226
↓ 1 callersFunction_run_onbuild_trigger
(trigger: str)
src/skillevaluator/tier3/harbor/adapter.py:2901
↓ 1 callersFunction_run_one
Execute one command and compare the marker against both checkers. With ``shell``, the command is run by that shell and scored as a native cal
scripts/script_invocation_differential.py:1271
↓ 1 callersFunction_run_rows
(run_dir: Path, selected_agent: str | None)
src/skillevaluator/tier3/history.py:149
↓ 1 callersMethod_run_safety
Run Safety check for supplementary coverage.
src/skillevaluator/validators/dependencies.py:725
↓ 1 callersFunction_run_sites
Every file the text runs (by interpreter, 'source', or as a command), at most ``MAX_RUN_SITES``.
src/skillevaluator/plugin_component_risk.py:2581
↓ 1 callersMethod_run_skillspector_once
Run one SkillSpector stage and validate its process/report contract.
src/skillevaluator/validators/security.py:3416
↓ 1 callersFunction_run_stop_on_pass_variant
Run up to ``n_concurrent`` cases at once, stopping each case's attempts on first pass. A case launches its next attempt only after the previous o
src/skillevaluator/tier3/harbor/runner.py:4986
↓ 1 callersFunction_run_truth_metadata
Read dataset/evaluator truth owned by the evaluated run, never live source.
src/skillevaluator/evaluation/tier3_report.py:4936
↓ 1 callersFunction_run_verdict_agents
Return the agents a multi-agent run verdict reads, as the canonical report picks them. An agent kept by --tolerate-baseline-timeouts is set aside
src/skillevaluator/tier3/result_display.py:130
↓ 1 callersFunction_run_verdict_from_agents
Return the multi-agent run verdict and, when it is incomplete, why. The verdict is read from the agents that ran their with-skill arm and is
src/skillevaluator/tier3/result_display.py:152
↓ 1 callersFunction_runner_shell_line
``(runner, line)`` when a package runner runs a shell command line. ``npx -c <line>``, ``npm exec --call <line>`` (also ``npm x``), and ``pnp
src/skillevaluator/validators/mcp_static.py:683
↓ 1 callersFunction_runs_apply_patch
Whether a shell call runs ``apply_patch`` on a patch (``apply_patch <<'PATCH'``, as Codex does).
src/skillevaluator/tier3/eval_core/plugin_signals.py:3212
↓ 1 callersMethod_runs_download
Whether one of ``facts`` runs a file one of them downloads; each pair of scripts is decided once.
src/skillevaluator/plugin_component_risk.py:3354
↓ 1 callersFunction_runs_fetched_variable
(text: str, fetched: frozenset[str])
src/skillevaluator/plugin_component_risk.py:2173
↓ 1 callersFunction_runs_here_string
(text: str, fetched: frozenset[str])
src/skillevaluator/plugin_component_risk.py:2167
↓ 1 callersFunction_runs_in_first_request
Whether a first-turn hook handler runs before the first request of a new session. UserPromptSubmit has no matcher. A SessionStart matcher is matc
src/skillevaluator/plugin_context_cost.py:123
↓ 1 callersFunction_runs_inline_code
Whether a shell's option prefix carries its inline-code option. Only the options before the first script operand are the shell's own. In ``ba
src/skillevaluator/tier3/eval_core/script_invocation.py:2505
↓ 1 callersMethod_runs_only_node_scripts
Whether every bundled script is a Node script. ``run_script`` is the call the script checks look for in the instructions. It names a
src/skillevaluator/validators/quality_score.py:284
↓ 1 callersFunction_runs_script_code
Whether a top-level assignment reads the command line or calls a function this file defines.
src/skillevaluator/tier3/generate_dataset.py:245
↓ 1 callersFunction_runtime_env
Normalize Harbor runtime env config to Harbor's ``environment.env`` shape.
src/skillevaluator/tier3/evals_config.py:782
↓ 1 callersFunction_runtime_env_toml_block
(runtime_env: dict[str, str] | None)
src/skillevaluator/tier3/harbor/adapter.py:2312
↓ 1 callersFunction_runtime_probe_error_message
Return an actionable diagnostic for an installed but unusable CLI.
src/skillevaluator/tier3/harbor/local_runtime.py:570
↓ 1 callersFunction_runtime_probe_sandbox_error_message
(agent: str, command: str, exc: Exception)
src/skillevaluator/tier3/harbor/local_runtime.py:580
↓ 1 callersFunction_runtime_seconds
(engine_result: dict[str, Any] | None)
src/skillevaluator/evaluation/tier3_report.py:5050
↓ 1 callersFunction_runtime_skill_fingerprint
Hash the agent-visible runtime tree independent of its directory name.
src/skillevaluator/tier3/harbor/adapter.py:917
↓ 1 callersFunction_runtime_skill_reset_lines
Replace fixed, effective-home, and project-ancestor skill roots.
src/skillevaluator/tier3/harbor/adapter.py:3838
↓ 1 callersFunction_runtime_source_is_bound_to_target
Return whether one staged runtime source is covered by the target digest.
src/skillevaluator/tier3/harbor/runner.py:203
↓ 1 callersFunction_safe
(value: str)
src/skillevaluator/tier3/nemo_gym/results.py:26
↓ 1 callersFunction_safe_endpoint_origin
Show only endpoint origin; custom paths can contain secret routing tokens.
src/skillevaluator/model_commands.py:59
↓ 1 callersFunction_safe_hostname
(parsed: Any)
src/skillevaluator/validators/mcp_static.py:428
← previousnext →7,501–7,600 of 24,741, ranked by callers