MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 3 callersFunction_trajectory_token_counter
Return a browser-safe token counter without inventing zero usage.
src/skillevaluator/tier3/harbor/report_data.py:660
↓ 3 callersFunction_trajectory_with_metric_refs
()
tests/tier3/test_behavior_evidence.py:496
↓ 3 callersFunction_tree
(root: Path, files: dict[str, Any])
tests/tier3/test_openclaw_prepare.py:33
↓ 3 callersFunction_trial_agent
(failed_arm=None)
tests/reporting/test_diagnostic_trial_comparisons.py:19
↓ 3 callersFunction_trial_case_id
Return the case a persisted trial ran. Harbor names a trial after the first 32 characters of its task directory. So the name before ``__`` is
src/skillevaluator/tier3/case_summary.py:214
↓ 3 callersFunction_trial_dir
(run_dir: Path, agent: str, variant_dir: str, trial: object)
src/skillevaluator/tier3/case_summary.py:916
↓ 3 callersFunction_trial_dirs
Resolve one logical attempt to its persisted physical or native-step artifacts.
src/skillevaluator/tier3/case_summary.py:931
↓ 3 callersFunction_trial_exception_details
Return the Harbor trial-root exception type and display reason, if present.
src/skillevaluator/tier3/harbor/collector.py:837
↓ 3 callersFunction_trial_roots
Bound directory enumeration and refuse links before opening any trial artifacts.
src/skillevaluator/tier3/plugin_evidence.py:169
↓ 3 callersFunction_trial_row
A verifier reward scoring 0.5 on every metric but *scores*, with *na* recorded as N/A.
tests/reporting/test_applicability_saved_runs.py:465
↓ 3 callersFunction_trusted_inline_code
Read executable code from trusted template sources, never report content. Retained CI reports support Tier 1 only. Their executable script and ev
ci/native_reports.py:60
↓ 3 callersFunction_unit_interval_score
Return a finite score in the documented 0.0 to 1.0 range.
src/skillevaluator/tier3/commands.py:1812
↓ 3 callersFunction_unlink_if_same_file
Unlink only the exact regular file created by the current operation.
src/skillevaluator/tier3/output_provenance.py:94
↓ 3 callersFunction_unresolved_value
Whether a resolved word holds a value the text bound but cannot settle.
src/skillevaluator/tier3/eval_core/script_invocation.py:1493
↓ 3 callersFunction_unsupported_or_load_error
(message: str)
src/skillevaluator/tier3/loose_parts.py:673
↓ 3 callersMethod_unsupported_rows
(self, source: NativePluginSource)
src/skillevaluator/tier3/plugin_native.py:1090
↓ 3 callersMethod_updateHoverStyles
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 3 callersFunction_use_goal_script
(module: ModuleType, monkeypatch: pytest.MonkeyPatch, responses, calls)
tests/tier3/test_judge_contracts.py:68
↓ 3 callersFunction_user
(text: str, *, sidechain: bool = False)
tests/tier3/test_f90_signals_handoff_prompt.py:44
↓ 3 callersMethod_usernames_from_identity
Extract candidate Linux usernames from an identity string. Handles the common shapes found in skill ``author`` frontmatter and CI env
src/skillevaluator/validators/security.py:3072
↓ 3 callersFunction_validate_and_sanitize_custom_compose
Reject Docker-host escape features and remove sidecar host ports. When Harbor runs multiple trials concurrently, each gets its own compose pr
src/skillevaluator/tier3/harbor/adapter.py:3443
↓ 3 callersFunction_validate_catalog_identity
(entry_id: object, path: object, content_type: object)
src/skillevaluator/embedding/registry.py:862
↓ 3 callersFunction_validate_config
(raw: dict[str, Any], config_path: Path)
src/skillevaluator/tier3/evals_config.py:240
↓ 3 callersFunction_validate_custom_dockerfile
Basic validation. Returns error message or None if OK.
src/skillevaluator/tier3/harbor/adapter.py:2513
↓ 3 callersFunction_validate_dataset_contents
Validate the contents of a dataset file.
src/skillevaluator/tier3/evals_spec.py:1047
↓ 3 callersFunction_validate_embedding_text
(text: object, *, full_body: bool)
src/skillevaluator/embedding/registry.py:795
↓ 3 callersFunction_validate_existing_scaffold_state
Validate authored dataset/config inputs before changing a private snapshot.
src/skillevaluator/tier3/commands.py:131
↓ 3 callersFunction_validate_fallback_file_exact
( path: Path, manifest: _FileManifest, *, final_mode: bool, )
src/skillevaluator/tier3/harbor/secure_copy.py:2268
↓ 3 callersFunction_validate_fallback_tree_exact_stable
( stage: Path, manifests: tuple[_TreeManifest, ...], *, child_final_modes: bool, root_mode
src/skillevaluator/tier3/harbor/secure_copy.py:1889
↓ 3 callersFunction_validate_file_node_exact
( parent: int, staged: _StagedNode, manifest: _FileManifest, *, final_mode: bool, )
src/skillevaluator/tier3/harbor/secure_copy.py:1641
↓ 3 callersFunction_validate_final_result_tree
Match the structural envelope enforced by the report JSON loader.
src/skillevaluator/tier3/harbor/runner.py:369
↓ 3 callersMethod_validate_frontmatter
Parse SKILL.md and validate YAML frontmatter against Pydantic schema.
src/skillevaluator/validators/schema.py:428
↓ 3 callersFunction_validate_hidden_final_tree
(node: _StagedNode, manifests: tuple[_TreeManifest, ...])
src/skillevaluator/tier3/harbor/secure_copy.py:1259
↓ 3 callersFunction_validate_json_tree
Validate decoded JSON iteratively so validation itself cannot recurse.
src/skillevaluator/tier3/harbor/report_data.py:362
↓ 3 callersFunction_validate_output_roots_outside_evaluator_sources
Reject output roots that would be copied back as evaluator source data.
src/skillevaluator/tier3/harbor/adapter.py:710
↓ 3 callersFunction_validate_runtime_discovery_env
(runtime_env: dict[str, str] | None)
src/skillevaluator/tier3/harbor/adapter.py:3820
↓ 3 callersFunction_validate_runtime_loader_env
(runtime_env: dict[str, str] | None)
src/skillevaluator/tier3/harbor/adapter.py:3828
↓ 3 callersFunction_validate_skill_root
(skill_path: Path)
src/skillevaluator/tier3/workflow.py:33
↓ 3 callersFunction_validate_windows_read_directory_handle
Require one opened Windows traversal component to be a plain directory.
src/skillevaluator/utils/secure_fs.py:800
↓ 3 callersFunction_validated_native_task_model
(task_dir: Path, data: dict[str, Any])
src/skillevaluator/tier3/harbor/adapter.py:5302
↓ 3 callersMethod_validated_pip_audit_report
Return resolved dependencies and skipped count for canonical pip-audit JSON.
src/skillevaluator/validators/dependencies.py:640
↓ 3 callersFunction_value_in_text
``value`` occurs in ``text`` as a whole token: ``AT-48`` is not in ``AT-4821``. An occurrence counts unless a letter continues a letter, or a dig
src/skillevaluator/tier3/eval_core/plugin_signals.py:4459
↓ 3 callersFunction_vendor_install
The text of the agent CLI install in Harbor's installer command.
tests/tier3/test_installed_agent_bootstrap.py:108
↓ 3 callersMethod_verdict_panel
(self, verdict: Verdict)
src/skillevaluator/reporting/console_ui.py:819
↓ 3 callersFunction_verifier_env_block
(runtime_env: dict[str, str] | None = None, indent: str = "")
src/skillevaluator/tier3/harbor/adapter.py:434
↓ 3 callersFunction_verifier_smoke_command
(dockerfile: str)
tests/test_harbor_runtime_skill_isolation.py:4289
↓ 3 callersFunction_visible_command_path
(command: str | Path)
src/skillevaluator/tier3/harbor/local_runtime.py:235
↓ 3 callersFunction_wait_for_bridge_server
Wait for ``bridge.serve`` to report its bound server, failing at once if its thread died first. Binding, the ready-file fsync and thread scheduli
tests/test_nvidia_build_bridge.py:269
↓ 3 callersMethod_wait_for_container_exec_ready
Wait until the GKE kubelet accepts exec streams and verify metadata server isolation.
src/skillevaluator/tier3/harbor/gke_environment.py:1161
↓ 3 callersFunction_windows_artifact_disk_budget
Return the current free bytes and an adaptive initial byte budget.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:1083
↓ 3 callersFunction_windows_existing_ancestor
Return the canonical nearest existing ancestor of a host path.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:1069
↓ 3 callersFunction_windows_kernel32
()
src/skillevaluator/utils/secure_fs.py:579
↓ 3 callersFunction_windows_raise_last_error
(message: str)
src/skillevaluator/utils/secure_fs.py:587
↓ 3 callersFunction_with_harness_statuses
A copy of ``trajectory`` whose results carry the harness's own per-call status. The status goes to ``extra.harness_status`` of every result whose
src/skillevaluator/tier3/plugin_trace_artifacts.py:381
↓ 3 callersFunction_with_skill_skipped
A baseline-only run (--skip-with-skill) deliberately never ran the with-skill arm.
src/skillevaluator/tier3/case_summary.py:81
↓ 3 callersFunction_workflow
()
tests/ci/test_image_build.py:40
↓ 3 callersFunction_workflow
()
tests/ci/test_evaluate_workflow.py:16
↓ 3 callersFunction_write
(path: Path, value: object)
tests/reporting/test_dashboard_activation_regressions.py:129
↓ 3 callersFunction_write_compare_summary
(run_dir: Path, agent: str, condition: str, summary: dict[str, object])
tests/test_tier3_compare.py:74
↓ 3 callersFunction_write_config
(skill_path: Path, mode: str)
tests/test_evals_config_branding.py:13
↓ 3 callersFunction_write_dataset_toml
Generate a minimal dataset.toml for the Harbor dataset.
src/skillevaluator/tier3/harbor/adapter.py:6176
↓ 3 callersFunction_write_dist
(root: Path, name: str, value: str)
tests/test_custom_execution_backends.py:413
↓ 3 callersFunction_write_job
Write a Harbor job with one trial per row: numeric ``reward.json`` plus the verifier's sidecar.
tests/reporting/test_applicability_saved_runs.py:477
↓ 3 callersFunction_write_json
(path: Path, value: dict[str, object])
tests/test_harbor_unscored_grading_policy.py:32
↓ 3 callersFunction_write_json
(path: Path, data: Any)
tests/tier3/test_view_results_feedback.py:52
↓ 3 callersMethod_write_line
(self, line: str)
src/skillevaluator/tier3/harbor/progress.py:400
↓ 3 callersMethod_write_line
(self, line: str)
src/skillevaluator/tier3/harbor/progress.py:899
↓ 3 callersFunction_write_metadata_skill
Write a skill whose frontmatter ``metadata`` is the given raw YAML value.
tests/validators/test_quality_score.py:137
↓ 3 callersFunction_write_one_case
(skill: Path, marker: str = "generated")
tests/test_evaluation_service.py:333
↓ 3 callersFunction_write_plugin
(root: Path, mcp_server: dict[str, object], hook: dict[str, object])
tests/validators/test_url_policy.py:428
↓ 3 callersFunction_write_results_trial
( tmp_path: Path, *, skill_name: str, trial_folder: str, trajectory: dict[str, object],
tests/tier3/test_generate_dataset_results.py:149
↓ 3 callersFunction_write_rule
(root: Path, filename: str, title: str, description: str)
tests/embedding/test_extractor.py:100
↓ 3 callersFunction_write_runtime_skill
(path: Path, package: str)
tests/test_harbor_runtime_skill_isolation.py:51
↓ 3 callersFunction_write_script_skill
(tmp_path: Path, script: str)
tests/validators/test_quality_score.py:1539
↓ 3 callersFunction_write_skill
(root: Path, apt_packages: str | None)
tests/test_harbor_apt_install_lines.py:48
↓ 3 callersFunction_write_skill
(tmp_path: Path)
tests/test_harbor_claude_refusal_fallback.py:116
↓ 3 callersFunction_write_trace_results
( skill: Path, results_root: Path, case_id: str, trajectory: dict, )
tests/tier3/test_dataset_grounding.py:33
↓ 3 callersFunction_write_workflow
(root: Path, name: str, title: str, description: str)
tests/embedding/test_extractor.py:107
↓ 3 callersFunction_yaml_comment_line_numbers
YAML `#` comments only. Scalar content, including block and quoted forms, is scanned.
src/skillevaluator/validators/security.py:350
↓ 3 callersFunction_yaml_error_detail
(exc: yaml.YAMLError)
src/skillevaluator/quality_config.py:298
↓ 3 callersMethodabandon_delivery
Prevent timeout cleanup from re-entering a stuck callback.
src/skillevaluator/tier3/harbor/local_environment.py:199
↓ 3 callersMethodactive
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 3 callersMethodadd_broken_source
Record a declared config source that cannot be loaded, with the finding that says why.
src/skillevaluator/plugin_mcp.py:152
↓ 3 callersFunctionadvisory_severity
(severity)
src/skillevaluator/deduplication/plugin/catalog_checks.py:28
↓ 3 callersFunctionagent_eval_case_selection_issue
Return why a run limited by ``max_cases`` cannot be publication evidence.
src/skillevaluator/reporting/base.py:1480
↓ 3 callersFunctionagent_eval_dimension_breakdown
Return the per-agent evidence the Tier 3 dimension gate reads. ``agent_eval_dimension_verdict`` folds this breakdown into the one verdict it
src/skillevaluator/reporting/base.py:1275
↓ 3 callersFunctionagent_eval_publication_evaluated_at
Return a canonical ISO evaluation timestamp, rejecting shaped text.
src/skillevaluator/reporting/base.py:751
↓ 3 callersFunctionagent_plugins_schema_version
Return the Agent Plugins version named by a ``$schema`` URL, else ``None``. The comparison is exact, as in Codex and Hermes: surrounding whitespa
src/skillevaluator/plugin_formats.py:35
↓ 3 callersMethodall_closed
(self)
tests/utils/test_secure_fs_platform.py:372
↓ 3 callersMethodall_mcp_declarations
All distinct launch declarations across effective, overridden and additional client views.
src/skillevaluator/plugin_inventory.py:121
↓ 3 callersMethodanalyze
( self, config: Any, *, source: str, file: str, display: str, dialect: HookDialect = CLAUDE_HOOKS
src/skillevaluator/plugin_component_risk.py:3434
↓ 3 callersFunctionanalyze_hook_config
Analyze a source using its native flat/nested hook dialect without executing it.
src/skillevaluator/plugin_config_risk.py:293
↓ 3 callersFunctionanalyze_settings
Check a bounded parsed Claude settings object, with source-row attribution.
src/skillevaluator/plugin_config_risk.py:327
↓ 3 callersFunctionatif_content_has_text
Check for meaningful text rather than media-only placeholders. Keep atif_content_text unchanged: other evidence sections still need safe [ima
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:50
↓ 3 callersMethodbinding
Read a new binding; reusing or shadowing a name is rejected.
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:945
↓ 3 callersFunctionbuild
(tmp_path: Path)
tests/validators/test_skillspector_hook_surface.py:237
↓ 3 callersFunctionbuild_eval_base_image
Pre-build a Docker base image with verifier and public-provider dependencies. Builds once and tags with a content hash so rebuilds only happen wh
src/skillevaluator/tier3/harbor/adapter.py:1498
↓ 3 callersFunctionbuild_insights
Return ``{"conclusions": [...], "recommendations": [...]}`` from the LLM. The function never raises: when the LLM is unavailable it simply return
src/skillevaluator/evaluation/insights_judge.py:658
↓ 3 callersFunctionbuild_metric_evidence_refs
Build compact source refs for LLM-judged metrics. The refs are intentionally small: downstream reports can cite the exact trajectory or eval-
src/skillevaluator/tier3/eval_core/atif_helpers.py:832
↓ 3 callersFunctionbuild_telemetry_view
Project native recorded telemetry with bounded traversal, text and bytes. At most 32 agents, 64 cases/agent and 128 scope groups per metric famil
src/skillevaluator/reporting/telemetry_view.py:284
← previousnext →3,101–3,200 of 24,741, ranked by callers