MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 1 callersFunction_admit_posix_directory
List one held directory between stable snapshots and admit its entries in name order.
src/skillevaluator/utils/secure_fs.py:1197
↓ 1 callersFunction_admit_windows_directory
List one pinned directory and admit its entries in name order, then require it unchanged.
src/skillevaluator/utils/secure_fs.py:1338
↓ 1 callersFunction_admit_windows_entry
Admit one entry while a no-follow handle pins it; return it when it is a directory to descend.
src/skillevaluator/utils/secure_fs.py:1358
↓ 1 callersMethod_advance
(self, character: str)
src/skillevaluator/tier3/harbor/stream_redaction.py:170
↓ 1 callersMethod_advance
(self, character: str)
src/skillevaluator/tier3/harbor/secure_docker_environment.py:442
↓ 1 callersFunction_advisory
(result)
src/skillevaluator/deduplication/plugin/commands.py:64
↓ 1 callersFunction_agent
( name: object, raw: object, index: int, budget: _Budget, warnings: list[NativeNotice] )
src/skillevaluator/reporting/native_evidence.py:401
↓ 1 callersFunction_agent_arm
( agent: str, decision: Any, source: Any, base_import_path: str | None )
src/skillevaluator/tier3/harbor/loose_parts_arm.py:175
↓ 1 callersFunction_agent_attempts
(agent: Mapping[str, Any], known_cases: set[str])
src/skillevaluator/tier3/token_usage.py:222
↓ 1 callersFunction_agent_coverage_lines
List each agent arm that ran but failed, with its scored attempts and first recorded failure.
src/skillevaluator/reporting/benchmark.py:1535
↓ 1 callersFunction_agent_eval_bounded_fingerprint_value
Project malformed evidence into a small deterministic fingerprint shape.
src/skillevaluator/reporting/base.py:1591
↓ 1 callersFunction_agent_eval_candidate_rank
( result: ValidationResult, payload: dict[str, Any], )
src/skillevaluator/reporting/base.py:1735
↓ 1 callersFunction_agent_eval_consistent_count_field
Return a duplicated non-negative count only when persisted values agree.
src/skillevaluator/reporting/base.py:641
↓ 1 callersFunction_agent_eval_evaluated_at_rank
(value: object)
src/skillevaluator/reporting/base.py:1725
↓ 1 callersFunction_agent_eval_fingerprint_key
Return a bounded JSON-safe description of one sampled mapping key.
src/skillevaluator/reporting/base.py:1573
↓ 1 callersFunction_agent_eval_payload
Return the payload the score and run-detail lines describe. A card reports one live evaluation, so one payload is rendered: the strongest Tie
src/skillevaluator/reporting/benchmark.py:1122
↓ 1 callersFunction_agent_eval_payload_fingerprint
Return a stable tie-breaker without fully serializing rejected evidence.
src/skillevaluator/reporting/base.py:1691
↓ 1 callersFunction_agent_eval_publication_aggregates_consistent
Require duplicated publication summaries to match the selected agent.
src/skillevaluator/reporting/base.py:1367
↓ 1 callersFunction_agent_label
( name: str, agent: dict[str, Any], private_labels: tuple[str, ...] = (), )
src/skillevaluator/reporting/benchmark.py:1441
↓ 1 callersFunction_agent_label
(name: object, index: int)
src/skillevaluator/reporting/native_evidence.py:74
↓ 1 callersFunction_agent_model
(name: str, info: dict[str, Any], run_config: dict[str, Any] | None)
src/skillevaluator/evaluation/tier3_report.py:4864
↓ 1 callersFunction_agent_model_for_display
Return the resolved user-facing model for an agent when available.
src/skillevaluator/tier3/harbor/report.py:207
↓ 1 callersFunction_agent_model_label
( agent: str, harbor_result: dict[str, Any], agents_data: dict[str, dict[str, Any]], )
src/skillevaluator/tier3/harbor/report.py:229
↓ 1 callersFunction_agent_output_roots
Return output roots established by trusted trial configuration. The standard trial output root is fixed. Only an explicitly configured absolu
src/skillevaluator/tier3/eval_core/checks.py:6040
↓ 1 callersFunction_agent_result
A parent ``Agent`` result as Harbor 0.24 records it for Claude Code (saved check-19 live run).
tests/tier3/test_f90_signals_handoff.py:253
↓ 1 callersFunction_agent_runtime_failure_pattern_start
(value: str)
src/skillevaluator/tier3/harbor/collector.py:211
↓ 1 callersFunction_agent_table_label
( name: str, agent: dict[str, Any], private_labels: tuple[str, ...] = (), )
src/skillevaluator/reporting/benchmark.py:1455
↓ 1 callersFunction_agent_timeout_seconds
Return the agent timeout budget named by a Harbor ``AgentTimeoutError`` message.
src/skillevaluator/tier3/harbor/collector.py:7282
↓ 1 callersFunction_agent_usage
( agent: Mapping[str, Any], unexecuted: frozenset[str], dataset_cases: frozenset[str] )
src/skillevaluator/tier3/token_usage.py:497
↓ 1 callersFunction_agents
(value: Any, config_path: Path)
src/skillevaluator/tier3/evals_config.py:437
↓ 1 callersFunction_aggregate_condition_overall
Aggregate a condition consistently with lift and canonical reports. Mixed metric contracts use the logical attempt overall: the standard-metric
src/skillevaluator/tier3/harbor/collector.py:5782
↓ 1 callersFunction_aggregate_execution
Aggregate execution status while retaining hidden child error counts. Published error strings are deduplicated for display, but the total is an
src/skillevaluator/tier3/harbor/collector.py:7189
↓ 1 callersFunction_aggregate_rubric_score
Return a local 0-100 score using schema importance weights. High, medium, and low importance criteria carry weights 3, 2, and 1. The result i
src/skillevaluator/validators/rubric_eval.py:206
↓ 1 callersFunction_aggregate_trial_name
Build a deterministic, attempt-readable, filesystem-safe trial name.
src/skillevaluator/tier3/harbor/runner.py:4360
↓ 1 callersFunction_aggregate_weighted
Compute weighted with_skill / baseline / parts for one dimension.
src/skillevaluator/evaluation/dimension_judge.py:239
↓ 1 callersFunction_alias
(site: Path, route: str, receipt: dict)
ci/publish_reports.py:464
↓ 1 callersFunction_alias_chain
Return frontmatter whose description is a list of 2**levels items built from chained aliases.
tests/tier3/test_decoys.py:611
↓ 1 callersFunction_alias_dag
(levels: int)
tests/utils/test_structured_data.py:21
↓ 1 callersFunction_alias_dag
Return frontmatter whose *field* is a list built from chained YAML aliases.
tests/embedding/test_extractor.py:676
↓ 1 callersFunction_aliased_harbor_value
( harbor_raw: dict[str, Any], *, canonical: str, alias: str, config_path: Path, )
src/skillevaluator/tier3/evals_config.py:366
↓ 1 callersFunction_all_commands
()
tests/test_skillevaluator_skill.py:51
↓ 1 callersMethod_all_patterns_match
Check if all required patterns match the content.
src/skillevaluator/validators/license.py:614
↓ 1 callersFunction_all_uses
Every action reference: step-level actions and job-level reusable workflows.
tests/test_ci_workflows.py:84
↓ 1 callersFunction_allocate_unused_name
(parent_descriptor: int, prefix: str)
src/skillevaluator/tier3/harbor/secure_copy.py:1367
↓ 1 callersFunction_allow_shapes
The approval output shapes (``claude``, ``cursor``) ``text`` emits with an allow value. ``#`` comments are not read (a comment that mentions the
src/skillevaluator/plugin_component_risk.py:1901
↓ 1 callersFunction_allow_value_re
An allow value written apart from its key: a string literal (``'allow'``, ``\\"allow\\"``), a shell assignment (``DECISION=allow``), or a jq argum
src/skillevaluator/plugin_component_risk.py:206
↓ 1 callersMethod_allowed_skill_dirs
Allowed skill-root subdirectories: built-in defaults plus per-repo extras. Recognised skill-root subdirectories (``DEFAULT_ALLOWED_SKILL_DIRS
src/skillevaluator/validators/schema.py:1199
↓ 1 callersFunction_allowed_workspace_skills
( expected_skill: str, workspace_skill_names: list[str] | None, workspace_mode: str, acceptabl
src/skillevaluator/tier3/eval_core/checks.py:5394
↓ 1 callersFunction_allows_missing_evals_json_for_native_custom_only
(skill_path: Path)
src/skillevaluator/tier3/commands.py:1279
↓ 1 callersFunction_always_on_rule
Return ``(body, None)`` for a rule that applies to every task, or ``(None, reason)``. For harnesses whose only rules channel is always on: a scop
src/skillevaluator/tier3/plugin_native.py:1188
↓ 1 callersFunction_ancestor_depth
Count the levels from *path* up to *ancestor*, matching by file identity. A lexical comparison misses a case-variant spelling of the same directo
src/skillevaluator/tier3/decoys.py:78
↓ 1 callersFunction_anchor_slug
(*parts: object)
src/skillevaluator/reporting/html.py:2015
↓ 1 callersFunction_and_list_command
(count: int)
tests/tier3/test_apply_patch_security.py:1212
↓ 1 callersMethod_animateOptions
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunction_annotate
( inventory: PluginInventory, manifest: PluginManifestFile, section: str, rows: Iterable[DependencyRow] )
src/skillevaluator/plugin_dependency_records.py:59
↓ 1 callersFunction_anthropic_base_url
(environ: Mapping[str, str])
src/skillevaluator/provider_config.py:694
↓ 1 callersFunction_anthropic_max_tokens
Clamp ``max_tokens`` to the Anthropic SDK's per-model non-streaming limit. The SDK refuses a non-streaming request above that limit (8,192 for
src/skillevaluator/inference/client.py:139
↓ 1 callersFunction_anthropic_message
(index: int, model: str, text: str)
tests/validators/test_rubric_eval.py:1285
↓ 1 callersFunction_anthropic_message_from_events
(events: list[dict[str, Any]])
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1411
↓ 1 callersFunction_anthropic_message_to_chat
(message: dict[str, Any])
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1792
↓ 1 callersFunction_anthropic_served_model
Return the model that wrote an Anthropic reply when it can differ from the requested one. A server-side ``fallback`` block names the model that t
src/skillevaluator/inference/client.py:371
↓ 1 callersFunction_anthropic_tool_calls
(tool_calls: list[dict[str, Any]])
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1898
↓ 1 callersFunction_anthropic_tool_choice_to_chat
(choice: Any)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1494
↓ 1 callersFunction_anthropic_tool_result_to_chat
(block: dict[str, Any])
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1845
↓ 1 callersFunction_anthropic_tool_to_chat
(tool: Any)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1852
↓ 1 callersFunction_anthropic_tool_use_to_chat
(block: dict[str, Any])
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1827
↓ 1 callersFunction_anthropic_url
()
src/skillevaluator/tier3/harbor/templates/eval.py:1638
↓ 1 callersMethod_any_pattern_matches
Check if any exclusion pattern matches.
src/skillevaluator/validators/license.py:625
↓ 1 callersFunction_api_root
(value: str)
ci/model_config.py:62
↓ 1 callersFunction_append_case
(suite: ET.Element, case: _Case)
src/skillevaluator/reporting/junit_reporter.py:391
↓ 1 callersFunction_append_responses_tool_call
(messages: list[dict[str, Any]], message: dict[str, Any])
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1601
↓ 1 callersFunction_append_section_with_budget
( parts: list[str], title: str, body: str, max_chars: int, *, section_limit: int | Non
src/skillevaluator/tier3/eval_core/atif_helpers.py:342
↓ 1 callersFunction_apply_arithmetic_assignments
What the assignments in a command's arithmetic leave. The variable ends as a number, unchanged after an error, or never read because ksh abor
src/skillevaluator/tier3/eval_core/script_invocation.py:2298
↓ 1 callersFunction_apply_engine_failure
Carry a failed engine status (for example a backend launch failure) into the report payload. Scores from the trials that did complete are kept; o
src/skillevaluator/evaluation/tier3_report.py:601
↓ 1 callersFunction_apply_eval_bindings
Leave unsettled what ``eval`` may bind: its words run again as commands here. This shell expands the words first, so ``eval "$code"`` runs what `
src/skillevaluator/tier3/eval_core/script_invocation.py:1276
↓ 1 callersFunction_apply_final_file_mode
(descriptor: int, mode: int)
src/skillevaluator/tier3/harbor/secure_copy.py:2429
↓ 1 callersFunction_apply_final_tree_modes
(descriptor: int, manifests: tuple[_TreeManifest, ...])
src/skillevaluator/tier3/harbor/secure_copy.py:1233
↓ 1 callersFunction_apply_harbor_result_case_identity
Apply trusted staged identity, or retain legacy reward fallback.
src/skillevaluator/tier3/harbor/collector.py:4304
↓ 1 callersFunction_apply_manifest
( staged_descriptor: int, manifest: _TreeManifest, *, overwrite: bool, )
src/skillevaluator/tier3/harbor/secure_copy.py:1047
↓ 1 callersFunction_apply_manifest_fallback
(stage: Path, manifest: _TreeManifest)
src/skillevaluator/tier3/harbor/secure_copy.py:1790
↓ 1 callersFunction_apply_modes_fallback
(stage: Path, manifests: tuple[_TreeManifest, ...])
src/skillevaluator/tier3/harbor/secure_copy.py:1813
↓ 1 callersFunction_apply_patch_heredoc_text
Return a shell *command* without the file header lines of each heredoc body fed to apply_patch. A body counts when its heredoc is the only one de
src/skillevaluator/tier3/eval_core/checks.py:1605
↓ 1 callersFunction_apply_patch_path_text
Return the action text of a shell apply_patch call for the command-text path checks. ``_apply_patch_protected_entry`` already checked every file
src/skillevaluator/tier3/eval_core/checks.py:1636
↓ 1 callersFunction_apply_patch_prefix_is_unsettled
Return whether the last cd/pushd/popd in a shell apply_patch *prefix* may not set its directory. Operators and keywords inside closed quoted stri
src/skillevaluator/tier3/eval_core/checks.py:1330
↓ 1 callersFunction_apply_patch_protected_entry
Return the protected-path entry the first protected apply_patch file header targets.
src/skillevaluator/tier3/eval_core/checks.py:1491
↓ 1 callersFunction_apply_saved_feedback
Show the report feedback a completed run saved, with no LLM call. The case-findings pass still runs without the LLM, on the run's own ``what_
src/skillevaluator/evaluation/tier3_report.py:4303
↓ 1 callersFunction_apply_set_options
Turn allexport on or off as ``set -a`` / ``set -o allexport`` and their ``+`` forms do, reading options up to the first operand or ``--``.
src/skillevaluator/tier3/eval_core/script_invocation.py:981
↓ 1 callersFunction_approval_hooks
(command: str)
tests/validators/test_plugin_static_risk_review_fixes.py:471
↓ 1 callersFunction_apt_packages
The packages named by a generated Dockerfile's ``apt-get install`` command.
tests/tier3/test_installed_agent_bootstrap.py:373
↓ 1 callersFunction_argument_failures
( args: Mapping[str, Any], rule: Mapping[str, Any], budget: list[float] )
src/skillevaluator/tier3/eval_core/plugin_signals.py:3961
↓ 1 callersFunction_arguments
(tool_call: Mapping[str, Any])
src/skillevaluator/tier3/eval_core/plugin_signals.py:1748
↓ 1 callersFunction_argv
(*argv: str)
tests/tier3/test_security_home_paths.py:75
↓ 1 callersFunction_argv_flag_hits
Options split across adjacent argv tokens (``["--sandbox", "danger-full-access"]``); the Codex-only forms count only after a codex token, or when
src/skillevaluator/validators/mcp_static.py:3283
↓ 1 callersFunction_arithmetic_close
Where the ``((`` before ``start`` is closed as an arithmetic command. bash, zsh, ksh and mksh read ``((`` as arithmetic when the parenthesis that
src/skillevaluator/tier3/eval_core/script_invocation.py:2171
↓ 1 callersFunction_arithmetic_texts
The arithmetic a command evaluates, where an assignment binds as it does in ``((...))``: that body, the words after ``let``, and with ``expans
src/skillevaluator/tier3/eval_core/script_invocation.py:2259
↓ 1 callersFunction_arm_cell
(arm: object)
src/skillevaluator/reporting/tier3_token_usage.py:93
↓ 1 callersFunction_arm_expected
Return the attempts the arm should have run: its recorded expected attempts plus recovered ones. A timed-out attempt that a later attempt recover
src/skillevaluator/tier3/token_usage.py:284
↓ 1 callersFunction_arm_failures
Name how each agent missed the run verdict, or None for a baseline-only run. ``both`` is an agent that completed neither arm, ``with_skill`` one
src/skillevaluator/tier3/result_display.py:184
↓ 1 callersFunction_arm_summary
(attempts: list[_Attempt], *, expected: int, not_run: bool)
src/skillevaluator/tier3/token_usage.py:344
↓ 1 callersFunction_arrow_cell
( with_value: float | None, without_value: float | None, fmt: str, delta_fmt: str, *, comparable: bool = T
src/skillevaluator/tier3/case_summary.py:701
← previousnext →5,501–5,600 of 24,741, ranked by callers