MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 33 callersMethodbuild_from_directory
Discover and embed a complete replacement selection. Successful builds replace prior entries, dimensions and coverage, even for an em
src/skillevaluator/embedding/registry.py:230
↓ 33 callersFunctioncheck_security
Scan actions, with output ownership from trusted agent_workdir configuration.
src/skillevaluator/tier3/eval_core/checks.py:6710
↓ 33 callersFunctionfetch_model_records
Fetch and normalize the selected provider's ``/models`` catalog.
src/skillevaluator/model_catalog.py:356
↓ 33 callersFunctionload_dashboard_path
Load one JSON report, a retained run, or a tree of reports and runs. Native run subtrees are consumed once and pruned, so their mirrored JSON
src/skillevaluator/reporting/dashboard_data.py:807
↓ 32 callersMethod_run_docker_compose_command
Run compose with sensitive values only in child env or stdin.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:3079
↓ 32 callersFunctionadvisory_skip_result
Return a non-blocking Tier 3 result recording why Tier 3 did not produce data. Mirrors the advisory Tier 3 behavior for an explicitly requested
src/skillevaluator/evaluation/tier3_report.py:355
↓ 32 callersMethodcompletions
(self, *_args)
tests/tier3/test_dataset_grounding.py:282
↓ 32 callersFunctionexample
(tmp_path, *, result=None)
tests/ci/test_report_views.py:18
↓ 32 callersFunctionis_cleanly_skipped
Return whether a result records a non-failing skipped execution.
src/skillevaluator/reporting/base.py:433
↓ 32 callersMethodload_catalog
Load and validate a versioned local embedding catalog.
src/skillevaluator/embedding/registry.py:615
↓ 32 callersFunctionmake_chunk
Factory for ContentChunk objects with sensible defaults.
tests/deduplication/conftest.py:48
↓ 32 callersFunctionpublication_source_digest
Return the canonical digest of one stable, safe source snapshot. Directories use a descriptor-anchored manifest on supported POSIX systems an
src/skillevaluator/publication_identity.py:631
↓ 31 callersFunction_finite_float
Return a JSON-safe finite number, rejecting booleans and non-finite floats.
src/skillevaluator/evaluation/tier3_report.py:127
↓ 31 callersFunction_select
(selection: SimpleNamespace, **kwargs: object)
tests/tier3/test_decoys.py:77
↓ 31 callersMethodflush
(self)
tests/test_harbor_output_provenance.py:200
↓ 31 callersFunctionrun_record
()
tests/ci/test_publish_reports.py:407
↓ 31 callersFunctionstrip_terminal_controls
Remove terminal escape sequences and control characters other than tab, LF and CR. Rich only strips BEL, BS, VT, FF and CR, so without this untru
src/skillevaluator/utils/rich_markup.py:26
↓ 31 callersMethodtitle
Get title from frontmatter.
src/skillevaluator/models/rules.py:171
↓ 31 callersFunctionwrite_skill
Factory fixture for creating test skill directories with SKILL.md.
tests/embedding/conftest.py:12
↓ 30 callersFunction_render_agent_payload
(payload: dict)
tests/reporting/test_unified_tier3_report.py:147
↓ 30 callersFunction_run
(root: Path, name: str = "20260709_120000")
tests/test_results_location_legacy_compatibility.py:21
↓ 30 callersFunction_tier
(page, name: str)
tests/reporting/test_report_browser.py:476
↓ 30 callersFunctionescape_markup
Return ``text`` made safe to interpolate into Rich markup. Terminal escape sequences and control characters other than tab and newline are re
src/skillevaluator/utils/rich_markup.py:46
↓ 30 callersMethodkill
(self)
tests/test_harbor_local_mode.py:6066
↓ 30 callersMethodnormalize
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 30 callersFunctiono
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 30 callersMethodread
Return the response body bytes.
tests/conftest.py:603
↓ 30 callersMethodstage_arm
Stage one arm's plugin content into ``env_dir`` (a task's ``environment/``) with the runner's code. ``without_skill`` stages nothing, ``sum_o
src/skillevaluator/tier3/loose_parts.py:356
↓ 29 callersFunction_case
An agentskills.io entry (prompt, expected_output, assertions), normalized as the loader does.
tests/tier3/test_dataset_baseline_fairness.py:40
↓ 29 callersFunction_cf
Shorthand to build a CollectedFile from content string.
tests/deduplication/utils/test_chunker.py:21
↓ 29 callersFunction_command
(command: str, **extra: object)
tests/validators/test_plugin_static_risk_review_fixes.py:73
↓ 29 callersFunction_load_harbor_template_module
()
tests/tier3/test_behavior_evidence.py:117
↓ 29 callersFunction_pair_script
(responses: list[tuple[str | None, str | None]], calls: list[str])
tests/tier3/test_judge_result_semantics.py:215
↓ 29 callersFunction_report
(name: str = _LEGACY_REPORT)
tests/validators/test_skillspector_reference_diagnostics.py:71
↓ 29 callersFunction_report
(name: str)
tests/validators/test_skillspector_hook_surface.py:185
↓ 29 callersFunction_require
(condition, message)
ci/feedback.py:87
↓ 29 callersFunction_rows
(skill: Path)
tests/harness/test_overlay.py:335
↓ 29 callersFunction_write_legacy_run_config
(run: Path, *agents: str)
tests/test_results_location_legacy_compatibility.py:27
↓ 29 callersFunctionbuild_user_prompt
Build the user prompt from cluster member chunks. Every member is rendered with its full text. When the request (``SYSTEM_PROMPT`` plus this
src/skillevaluator/deduplication/intra_skill/llm_analyzer.py:66
↓ 29 callersFunctionextract_from_skill
Extract name + description from a skill directory's SKILL.md.
src/skillevaluator/embedding/extractor.py:167
↓ 29 callersFunctionl
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 29 callersFunctionn
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 29 callersFunctionplugin_inventory_for_root
Compose every real client declaration and fallback view without public activation.
src/skillevaluator/plugin_inventory.py:421
↓ 29 callersFunctionstrict
(monkeypatch)
tests/tier3/test_judge_policies.py:75
↓ 29 callersMethodwait
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 28 callersFunction_collect
(tmp_path: Path, launch_errors: list[str], *, agents: list[str] | None = None, **kwargs: Any)
tests/tier3/test_baseline_timeouts.py:205
↓ 28 callersFunction_local_environment
( tmp_path: Path, *, persistent_env: dict[str, str] | None = None )
tests/test_harbor_local_mode.py:85
↓ 28 callersFunctionload_harbor_eval_template
Load the standalone Harbor verifier template module dynamically.
tests/conftest.py:576
↓ 28 callersFunctionresolve_run_status
(results: Iterable[ValidationResult], *, agent_eval=None, expected_skill_name=None)
src/skillevaluator/reporting/run_status.py:149
↓ 28 callersMethodsave_catalog
Persist a validated, versioned local embedding catalog.
src/skillevaluator/embedding/registry.py:564
↓ 28 callersFunctionsecurity_check
(request: pytest.FixtureRequest)
tests/tier3/test_destructive_command_paths.py:18
↓ 28 callersFunctionskill_facts
Compute the facts profile rules refer to (see profiles README). ``catalog`` describes the other skills installed alongside this one; without it t
src/skillevaluator/harness/overlay.py:429
↓ 27 callersFunction_client
(vectors: list[list[float]] | None = None)
tests/embedding/test_catalog.py:35
↓ 27 callersFunction_inventory
(root: Path)
tests/test_openclaw_plugin_discovery.py:45
↓ 27 callersFunction_write_report
( path: Path, *, skill: str = "calculator", model: str = "model-a", harness: str = "openco
tests/reporting/test_dashboard_app.py:30
↓ 27 callersMethodcompletions
Send a chat completion request and return the response text. Raises :class:`LLMClientError` when the response is empty. On success :a
src/skillevaluator/inference/client.py:681
↓ 27 callersFunctiondiscover_and_extract
Auto-discover content items under a directory and extract entries. Args: root: Root directory to scan. content_type: One of "skil
src/skillevaluator/embedding/extractor.py:248
↓ 27 callersFunctionload_bounded_yaml
Parse one YAML document after bounded event and graph validation. A duplicate mapping key is a syntax error, unless ``last_key_wins`` keeps t
src/skillevaluator/utils/structured_data.py:229
↓ 27 callersMethodmerge
Merge another result into this one, combining all fields.
src/skillevaluator/models/result.py:717
↓ 27 callersMethodparse_args
(self, ctx: click.Context, args: list[str])
src/skillevaluator/tier_group.py:21
↓ 27 callersFunctionrender_evaluation_result
Render persisted engine truth, aggregating only canonical score components. The case-first summary always leads; ``verbose`` adds the per-evaluat
src/skillevaluator/tier3/result_display.py:1154
↓ 26 callersFunction_activations
(traj: dict[str, Any])
tests/tier3/test_plugin_signals.py:79
↓ 26 callersFunction_is_link
(metadata: os.stat_result)
src/skillevaluator/tier3/harbor/secure_copy.py:206
↓ 26 callersFunction_write_authoritative_multistep_result
( job_dir: Path, trial_name: str, *, aggregate: dict[str, object], step_rewards: list[dict
tests/test_issue55_collector_truth.py:133
↓ 26 callersFunction_write_skill
(parent: Path, name: str, *, body: str | None = None)
tests/test_precommit.py:50
↓ 26 callersFunctionapi
()
tests/reporting/test_private_runtime_projection.py:13
↓ 26 callersFunctionbuild_loose_parts
Plan the loose-parts arm for ``agent`` from the native plugin snapshot. Never raises for an unsupported agent or load mode, or a plugin with
src/skillevaluator/tier3/loose_parts.py:586
↓ 26 callersMethodgetDatasetMeta
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 26 callersMethodmerge_with_prefix
Merge another result, prefixing all errors/warnings with skill name.
src/skillevaluator/models/result.py:755
↓ 26 callersFunctionmetric_set_for_reward
Return the metric-set label and metric order used by a reward payload.
src/skillevaluator/tier3/harbor/metrics.py:443
↓ 26 callersFunctionprepare_runtime_package
Build the prepared package from the selected manifest's effective rows. ``expected`` is the identity the caller analyzed; a different root, m
src/skillevaluator/tier3/plugin_prepare.py:141
↓ 26 callersMethodremove
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 26 callersFunctionresolve_latest_results
Return the best available ``latest`` results path for read workflows. Prefers each root's valid ``latest`` symlink. The link must resolve to an
src/skillevaluator/tier3/results_location.py:1069
↓ 25 callersFunction_assert_error_result
(result: dict)
tests/tier3/test_judge_result_semantics.py:254
↓ 25 callersFunction_flat_arms
(cases: list[str], none: float, skills: float, loose: float, full: float)
tests/tier3/test_private_ladder_stats.py:63
↓ 25 callersFunction_issue
( field_name: str, error: str, message: str, level: IssueLevel = "error", suggestion: str = "" )
src/skillevaluator/plugin_format_schema.py:62
↓ 25 callersFunction_resolved_shell_arg
(arg: str, assignments: dict[str, str])
src/skillevaluator/tier3/eval_core/script_invocation.py:32
↓ 25 callersFunction_write_successful_summary
(run: Path, *, agent: str = "opencode")
tests/test_results_location_legacy_compatibility.py:105
↓ 25 callersFunctionadapter
()
tests/ci/test_feedback.py:26
↓ 25 callersFunctioncopytree_secure
Copy a link-free authored tree and publish it transactionally. Every non-ignored source entry must be a single-link regular file or a directo
src/skillevaluator/tier3/harbor/secure_copy.py:2113
↓ 25 callersFunctionmetric_value
Return a numeric metric value from a reward payload, if present.
src/skillevaluator/tier3/harbor/metrics.py:308
↓ 25 callersFunctionreplay
Answer each judge request with the next outcome and return the requested models.
tests/tier3/test_judge_policies.py:93
↓ 25 callersFunctionresult
(tier=1, **metadata)
tests/reporting/test_final_render_context.py:25
↓ 25 callersMethodservice_exec
Execute in main securely, or in an isolated POSIX sidecar.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:2903
↓ 25 callersFunctionsummarize_tier1
Build the Tier 1 section rows from its validator results. When *lineup* is given, the per-check ticker stays in the completed section with ea
src/skillevaluator/reporting/console_ui.py:1167
↓ 24 callersMethod_add
(self, component: PluginMember)
src/skillevaluator/plugin_discovery.py:249
↓ 24 callersFunction_app
(paths: list[Path])
tests/reporting/test_dashboard_app.py:82
↓ 24 callersFunction_collect
(tmp_path: Path, **kwargs: object)
tests/test_harbor_collector_runtime_failures.py:1831
↓ 24 callersFunction_hooks
(event: str, *handlers: dict, matcher: str | None = None)
tests/validators/test_plugin_static_risk_review_fixes.py:77
↓ 24 callersFunction_load_verifier
Load and initialize the Harbor verifier template module in a temporary workspace.
tests/tier3/test_judge_failure_artifacts.py:52
↓ 24 callersFunction_project
(payload)
tests/reporting/test_insights_view.py:12
↓ 24 callersFunction_request
( url: str, payload: object | None = None, *, headers: dict[str, str] | None = None, )
tests/test_nvidia_build_bridge.py:238
↓ 24 callersFunction_scan
(target: Path, waiver_path: Path | None)
tests/validators/test_pii_waivers.py:71
↓ 24 callersFunction_skill
(tmp_path: Path, files: dict[str, str], name: str = "demo-skill")
tests/validators/test_dependency_advisory_findings.py:28
↓ 24 callersFunctiona
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 24 callersFunctioncompute_plugin_signals
Per-trial C3 ``plugin_signals`` for one ATIF trajectory, or ``None`` if unreadable. ``case`` is a dataset case entry or a :func:`plugin_case_spec
src/skillevaluator/tier3/eval_core/plugin_signals.py:4757
↓ 24 callersFunctiondocuments
(monkeypatch, *, eligible=True)
tests/ci/test_benchmark_publish.py:132
↓ 24 callersMethodemit
(self, _event)
tests/test_tier3_progress.py:1074
↓ 24 callersFunctionexternal_results_root
Return ``<root>/<skill-name>`` for a global or CLI results root.
src/skillevaluator/tier3/results_location.py:64
↓ 24 callersMethodwait
(self, timeout: float | None = None)
tests/reporting/test_dashboard_cli.py:32
← previousnext →201–300 of 24,741, ranked by callers