MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 1 callersFunction_editor
A text editor tool call (OpenHands ``str_replace_editor``, ``str_replace_based_edit_tool``).
tests/tier3/test_security_home_paths.py:70
↓ 1 callersFunction_effect
One word per case on what the skill changed: pass rate first, then mean score.
src/skillevaluator/tier3/case_summary.py:546
↓ 1 callersFunction_effect_diagnostics
( effect: Text, with_case: Mapping[str, Any] | None, without_case: Mapping[str, Any] | None, s
src/skillevaluator/tier3/case_summary.py:531
↓ 1 callersFunction_effective_compare_policy
Treat unversioned complete SkillEvaluator summaries as historical scores.
src/skillevaluator/tier3/commands.py:1785
↓ 1 callersFunction_effective_openai_base_url
(explicit_base_url: str | None)
src/skillevaluator/inference/client.py:70
↓ 1 callersFunction_embedded_json_objects
Return the complete top-level JSON objects embedded in prose. Each ``{`` or ``[`` starts a string-aware balanced span that is decoded as a wh
src/skillevaluator/inference/client.py:410
↓ 1 callersFunction_embedded_texts
(client: MagicMock)
tests/embedding/test_similarity_limits.py:317
↓ 1 callersFunction_emit_run_finished
(state: str, detail: str, *, include_artifacts: bool = True)
src/skillevaluator/tier3/harbor/runner.py:6122
↓ 1 callersFunction_emit_started_agents
()
src/skillevaluator/tier3/harbor/runner.py:6684
↓ 1 callersFunction_encode_final_result
Serialize one final result only when the browser/report loader can consume it.
src/skillevaluator/tier3/harbor/runner.py:386
↓ 1 callersFunction_encodes
(text: str, encoding: str)
src/skillevaluator/cli_help.py:72
↓ 1 callersFunction_endpoint
(url: str, base: str | None = None)
src/skillevaluator/validators/endpoint_resolution.py:155
↓ 1 callersFunction_endpoint_fingerprint
(url: str)
tests/embedding/test_catalog.py:31
↓ 1 callersFunction_ends_with
(words: list[str], end: int, phrases: tuple[tuple[str, ...], ...])
src/skillevaluator/evaluation/dataset_case_removal.py:73
↓ 1 callersFunction_enforce_allowed_model
Prevent an authenticated trial from turning the bridge into a general credential proxy.
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:349
↓ 1 callersFunction_enforce_report_payload_budget
Keep the complete self-contained payload within a hard serialized budget. Cardinality limits normally keep the payload comfortably below the cap.
src/skillevaluator/evaluation/tier3_report.py:1605
↓ 1 callersFunction_engine
(by_server: dict[str, dict[str, int]])
tests/tier3/test_mcp_proof.py:513
↓ 1 callersFunction_ensure_environment_env
Ensure staged native tasks populate [environment.env] with runtime environment variables.
src/skillevaluator/tier3/harbor/adapter.py:5655
↓ 1 callersFunction_ensure_native_metadata_entry_id
Ensure [metadata].entry_id is present in a native task.toml document.
src/skillevaluator/tier3/harbor/adapter.py:4939
↓ 1 callersFunction_ensure_native_skills_dir
Pin native tasks to the evaluator-owned runtime skill projection.
src/skillevaluator/tier3/harbor/adapter.py:5622
↓ 1 callersFunction_ensure_pre_agent_setup_healthcheck
(task_dir: Path, pre_agent_setup: list[str])
src/skillevaluator/tier3/harbor/adapter.py:5671
↓ 1 callersFunction_ensure_private_directories
(root_descriptor: int, manifest: _TreeManifest)
src/skillevaluator/tier3/harbor/secure_copy.py:967
↓ 1 callersFunction_ensure_runtime_env_and_pre_agent_setup
( task_dir: Path, *, runtime_env: dict[str, str] | None, pre_agent_setup: list[str] | None, )
src/skillevaluator/tier3/harbor/adapter.py:5700
↓ 1 callersFunction_ensure_skill_evaluator_verifier_env
Ensure staged native tasks forward configured public provider variables.
src/skillevaluator/tier3/harbor/adapter.py:5634
↓ 1 callersFunction_entry_declares_task_input
Return whether eval metadata should replace a native task's own input.
src/skillevaluator/tier3/harbor/adapter.py:4162
↓ 1 callersFunction_entry_names
Return the entry names of *folder*, or ``None`` when it cannot be listed.
src/skillevaluator/tier1/commands.py:175
↓ 1 callersMethod_entry_phrases
(self, k: int)
src/skillevaluator/evaluation/dataset_case_removal.py:436
↓ 1 callersFunction_entry_point_source
(entry_point: EntryPoint)
src/skillevaluator/tier3_environments.py:366
↓ 1 callersFunction_env_command
The command line ``env`` runs, given env's arguments: ``env -u X A=1 sh -c y`` runs ``sh -c y``.
src/skillevaluator/validators/mcp_static.py:1771
↓ 1 callersFunction_env_controls_endpoint
Whether a reference without a default decides the scheme, host, or port of ``expanded``.
src/skillevaluator/validators/mcp_static.py:902
↓ 1 callersFunction_env_errors
(trial_dir: Path, run_dir: Path)
src/skillevaluator/tier3/case_summary.py:989
↓ 1 callersFunction_env_int_or_none
Return environment variable *name* as an int, or ``None`` when unusable.
src/skillevaluator/tier3/eval_core/atif_helpers.py:246
↓ 1 callersFunction_env_mode_host_path_env_vars
Return a backend's host variables that name files, anchored before Harbor changes directory.
src/skillevaluator/tier3/harbor/runner.py:1392
↓ 1 callersFunction_env_option
``(takes a value, splits the value into arguments, the value attached to the word)`` for an env option.
src/skillevaluator/validators/mcp_static.py:1793
↓ 1 callersFunction_env_path
(name, default)
src/skillevaluator/tier3/harbor/templates/eval.py:641
↓ 1 callersFunction_env_path
(name, default)
src/skillevaluator/tier3/harbor/templates/custom_grader_runner.py:20
↓ 1 callersMethod_env_token_to_path
(self, token: str, exec_env: dict[str, str])
src/skillevaluator/tier3/harbor/local_environment.py:1775
↓ 1 callersFunction_environment
(ae: dict[str, Any] | None)
src/skillevaluator/reporting/benchmark.py:1643
↓ 1 callersFunction_environment
(workspace: Path, image: str, source: dict[str, str])
ci/run_tier1_image.py:76
↓ 1 callersFunction_environment_findings
(discovery: PluginDiscovery, reader: PluginRootReader)
src/skillevaluator/plugin_inventory.py:272
↓ 1 callersFunction_environment_kwarg_secret_policy_error
(value: dict[str, Any], *, env_mode: str | None)
src/skillevaluator/tier3/evals_config.py:673
↓ 1 callersFunction_environment_kwarg_shape_error
Validate bounded JSON shape iteratively so cycles/deep values fail closed.
src/skillevaluator/tier3/evals_config.py:561
↓ 1 callersFunction_epilog
(command: click.Command)
src/skillevaluator/cli_help.py:280
↓ 1 callersFunction_epilog_line
Color one pre-formatted epilog line (raw layout is preserved verbatim).
src/skillevaluator/cli_help.py:259
↓ 1 callersFunction_error_messages
(value: object)
src/skillevaluator/tier3/result_display.py:352
↓ 1 callersMethod_escape
(self)
src/skillevaluator/plugin_component_risk.py:1357
↓ 1 callersFunction_escape_backslashes_end
Where the backslashes end when the one at ``index`` belongs to an escape that is not a path. That is a ``\\n`` ``\\r`` or ``\\t`` escape, or the
src/skillevaluator/evaluation/tier3_feedback_chat.py:411
↓ 1 callersFunction_escape_surrogates
Return *text* with lone surrogates written as backslash escapes. A file name that is not valid UTF-8 arrives with surrogate escapes, which XM
src/skillevaluator/reporting/junit_reporter.py:64
↓ 1 callersFunction_escapes_name_character
Whether a ``\\u`` or ``\\x`` escape of a letter or digit (``\\u00e9``) is at ``index``, after its backslash.
src/skillevaluator/evaluation/tier3_feedback_chat.py:346
↓ 1 callersMethod_estimate_confidence
Estimate confidence level of a PII match based on line context. Returns 'high', 'medium', or 'low' confidence. High-confidence catego
src/skillevaluator/validators/security.py:5997
↓ 1 callersFunction_eval
(node)
src/skillevaluator/tier3/reference_skills/calculator/scripts/calc.py:28
↓ 1 callersMethod_eval
(cls, node: ast.AST, names: dict[str, Any])
src/skillevaluator/harness/overlay.py:800
↓ 1 callersFunction_evaluated_at
(ae: dict[str, Any] | None)
src/skillevaluator/reporting/benchmark.py:1574
↓ 1 callersFunction_evaluated_at_from_run
Return the persisted UTC evaluation time, never a guessed legacy date.
src/skillevaluator/evaluation/tier3_report.py:5002
↓ 1 callersFunction_evaluated_source
Return the validated evaluated-source identity recorded for this run. Every card shape needs a carrier for the identity, not just a completed
src/skillevaluator/reporting/benchmark.py:1619
↓ 1 callersFunction_evaluation_date
(value: str)
src/skillevaluator/reporting/benchmark.py:1578
↓ 1 callersFunction_evaluation_failure
Name what failed an unscored evaluation (environment, collector or judge) and the recorded reason.
src/skillevaluator/tier3/case_summary.py:285
↓ 1 callersFunction_evaluator_card
( metric: str, scores: dict[str, Any], *, label: str, rewards: list[dict[str, Any]], r
src/skillevaluator/evaluation/tier3_report.py:3265
↓ 1 callersFunction_evaluator_projection_selection
Resolve the evaluator projection and a post-copy selection signature.
src/skillevaluator/tier3/harbor/adapter.py:6504
↓ 1 callersFunction_every
Return the capture shared by every match, so a value quoted twice stays consistent.
tests/test_metrics_reference_docs.py:96
↓ 1 callersFunction_evidence
(result: dict)
ci/report_views.py:186
↓ 1 callersFunction_evidence
Run the baseline evidence check on the arm-level fixture, with the summary inputs it receives.
tests/tier3/test_baseline_timeouts.py:655
↓ 1 callersFunction_evidence_excerpt
(text: str, limit: int = _METRIC_EVIDENCE_EXCERPT_CHARS)
src/skillevaluator/tier3/eval_core/atif_helpers.py:611
↓ 1 callersFunction_evidence_section
Return the body of *title*'s section, or ``""`` when it is absent.
tests/tier3/test_behavior_evidence.py:566
↓ 1 callersFunction_evidence_unavailable_reason
(trials: list[dict[str, Any]])
src/skillevaluator/tier3/harbor/report.py:682
↓ 1 callersFunction_exact_entry
``None`` when ``parent`` holds an entry spelled exactly ``name``, else the entry that differs only in case. On a case-insensitive filesystem ``ls
src/skillevaluator/plugin_dependencies.py:440
↓ 1 callersFunction_exact_probability_fields
Serialize exact probability diagnostics without changing process-wide safety limits.
src/skillevaluator/tier3/harbor/collector.py:4949
↓ 1 callersMethod_exceeds_string_limit
Return whether *body* decodes to more than ``max_string_chars`` characters. A decoded character takes one to twelve input characters, so only
src/skillevaluator/utils/structured_data.py:340
↓ 1 callersFunction_excluded_os
Operating systems Hermes hides the skill on (it maps names, then prefix-matches sys.platform).
src/skillevaluator/harness/overlay.py:380
↓ 1 callersMethod_exec
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunction_exec_record
Check one record against its format; return its ``(is_error, evidence)``, or None if malformed. ``is_error`` is None while the record states no o
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:1312
↓ 1 callersMethod_exec_without_environment
( self, command: str, *, cwd: str | None, timeout_sec: int | None,
src/skillevaluator/tier3/harbor/secure_docker_environment.py:3422
↓ 1 callersFunction_execute_operation
Execute an operation with persisted state.
src/skillevaluator/tier3/reference_skills/task-list/scripts/tasks.py:389
↓ 1 callersFunction_executed_trajectory
Flatten only executed child references, preserving distinct ownership scopes. File references are resolved by the host adapter. Direct pure calle
src/skillevaluator/tier3/eval_core/plugin_signals.py:3023
↓ 1 callersFunction_execution_backend_name
Return the backend a run recorded; Harbor runs record none.
src/skillevaluator/evaluation/tier3_report.py:4901
↓ 1 callersFunction_expanded_python_imports
Return ``module.name`` forms for ``from module import name`` statements. Returns None when Python source is too deeply nested to parse.
src/skillevaluator/validators/skillspector_scope.py:916
↓ 1 callersFunction_expanded_tool_calls
One raw tool call as normalized calls: a Codex ``exec`` wrapper becomes the calls it made.
src/skillevaluator/tier3/eval_core/plugin_signals.py:2931
↓ 1 callersFunction_expected
(cases, thresholds)
src/skillevaluator/tier3/grader_policy.py:102
↓ 1 callersFunction_expected_artifact_refs
( ground_truth: str, expected_behavior: list[str], )
src/skillevaluator/tier3/eval_core/atif_helpers.py:773
↓ 1 callersFunction_expected_behavior_refs
(expected_behavior: list[str])
src/skillevaluator/tier3/eval_core/atif_helpers.py:804
↓ 1 callersFunction_expected_behaviors_verified
Return True when deterministic facts already settle every expected behavior. An expectation is settled when every checkable token it names — an
src/skillevaluator/tier3/eval_core/atif_helpers.py:1215
↓ 1 callersFunction_expected_step_trajectory_names
(trial_root: Path)
src/skillevaluator/tier3/harbor/collector.py:2849
↓ 1 callersFunction_explicit_harbor_timeout_values
Yield explicit task bases scaled by Harbor's global multiplier.
src/skillevaluator/tier3/harbor/runner.py:3568
↓ 1 callersMethod_explicit_skill_roots
Members of a format that loads only its declared skill roots (OpenClaw). No implicit ``skills/`` root is added. Declared roots are searched
src/skillevaluator/plugin_discovery.py:678
↓ 1 callersFunction_expose_fallback_tree_root_exact
(stage: Path, manifests: tuple[_TreeManifest, ...])
src/skillevaluator/tier3/harbor/secure_copy.py:1940
↓ 1 callersFunction_expose_named_file
( parent: int, name: str, manifest: _FileManifest, )
src/skillevaluator/tier3/harbor/secure_copy.py:2456
↓ 1 callersFunction_expose_named_tree_root
( parent_descriptor: int, name: str, manifest: _TreeManifest, )
src/skillevaluator/tier3/harbor/secure_copy.py:1298
↓ 1 callersFunction_extended_codex_program
Decode the extended grammar into ``(calls, owner, record_format)``, or None outside it. The statements are a literal command array, one `Promise.
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:1179
↓ 1 callersFunction_extract_agent_runtime_failures
(job_dir: Path)
src/skillevaluator/tier3/harbor/collector.py:1215
↓ 1 callersFunction_extract_calls
( trajectory: Mapping[str, Any], declared: Mapping[str, Sequence[str]], mcp_names: _McpNames,
src/skillevaluator/tier3/eval_core/plugin_signals.py:3110
↓ 1 callersFunction_extract_choice_content
(response: Any)
src/skillevaluator/inference/client.py:395
↓ 1 callersMethod_extract_issue_group_key
Extract a grouping key from a finding message. For SECURITY findings from skillspector, messages often include the matched code snipp
src/skillevaluator/reporting/html.py:2681
↓ 1 callersFunction_extract_mcp_env_refs
Extract all $VAR and ${VAR} references from a string or nested structure.
src/skillevaluator/tier3/harbor/adapter.py:1999
↓ 1 callersFunction_extract_regular_tar_archive
Validate a disk-backed container archive, then extract it privately.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:1206
↓ 1 callersFunction_extract_version
Return ``metadata.version`` only. The legacy top-level ``version`` field is intentionally ignored: prior to the optional-version proposal, ``
src/skillevaluator/validators/version.py:24
↓ 1 callersFunction_extracted_directory
Where an archive tool unpacks a download (``tar xzf x.tgz -C dir``, ``curl … | tar xz``), or ``None``.
src/skillevaluator/plugin_component_risk.py:2552
↓ 1 callersFunction_fail_execution_with_error
Fail one execution summary and publish ``error`` within the bounded error contract.
src/skillevaluator/tier3/harbor/collector.py:7170
↓ 1 callersFunction_failed_check_reason
Prefer a grader-written reason over the internal check key.
src/skillevaluator/evaluation/tier3_findings.py:156
↓ 1 callersFunction_failed_reasons
(reward: Mapping[str, Any])
src/skillevaluator/tier3/case_summary.py:971
↓ 1 callersFunction_failed_tier2_report
(output: Path, *, incomplete: bool = False)
tests/ci/test_run_tiers.py:741
← previousnext →6,201–6,300 of 24,741, ranked by callers