MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 16 callersMethodget
(self, path)
tests/ci/test_publish_reports.py:1389
↓ 16 callersMethodget_install_hint
Get installation hint for this tool.
src/skillevaluator/utils/tool_runner.py:316
↓ 16 callersFunctionget_skip_reason
Return a bounded, single-line reason safe for downstream renderers.
src/skillevaluator/reporting/base.py:445
↓ 16 callersFunctionload_bounded_json
Parse strict JSON and validate its expanded object graph iteratively. The lexical preflight (:func:`preflight_json_structure`) allows ``max_t
src/skillevaluator/utils/structured_data.py:414
↓ 16 callersFunctionmock_tool_available
Context manager to mock a tool as available.
tests/validators/test_security_validators.py:37
↓ 16 callersMethodnotifyPlugins
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 16 callersMethodof
(cls, skills: list[Path], base: Path | None = None)
src/skillevaluator/harness/overlay.py:601
↓ 16 callersFunctionresponses_to_chat_request
Translate an OpenAI Responses request to a Build Chat Completions request.
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1090
↓ 16 callersFunctionreward
(case="a", **overrides)
tests/tier3/test_metric_applicability_producer.py:15
↓ 16 callersFunctionsanitize_grader_text
Collapse whitespace, redact credentials, and optionally clamp grader text. Like the Tier 3 findings report, this redacts the exact values of conf
src/skillevaluator/evaluation/tier3_findings.py:120
↓ 16 callersFunctionscore_value
Return a finite score inside SkillEvaluator's documented 0..1 range.
src/skillevaluator/tier3/harbor/metrics.py:294
↓ 16 callersMethodsearch
(self, *args: Any)
tests/tier3/test_security_finding_evidence.py:4212
↓ 16 callersFunctionstamp_publication_evidence
Stamp results from one trusted built-in command wrapper.
src/skillevaluator/publication_evidence.py:114
↓ 16 callersFunctionsynthetic_trajectory_from_codex_txt
Reconstruct ATIF from Codex tee logs (``exec --json`` JSONL or OpenCode-shaped JSONL).
src/skillevaluator/tier3/eval_core/log_converters.py:985
↓ 16 callersMethodtime
(self)
tests/test_harbor_local_mode.py:212
↓ 16 callersMethodto_dict
Convert to dictionary for JSON serialization.
src/skillevaluator/models/result.py:245
↓ 16 callersFunctionu
(t,e,i,s)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 16 callersMethodvalidate
Run both security and PII scanning on skill(s).
src/skillevaluator/validators/security.py:3177
↓ 16 callersFunctionvalidate_plugin_case_fields
Return problems with the optional advisory plugin-signal fields of one case. Absent fields are fine. ``null`` is treated as absent. Every present
src/skillevaluator/tier3/eval_core/plugin_signals.py:1180
↓ 16 callersFunctionvalidate_report
(report: dict, *, use_llm: bool = False)
tests/validators/test_security_reference_caveats.py:46
↓ 15 callersFunction_catalog_data
()
tests/embedding/test_catalog.py:48
↓ 15 callersFunction_collected
(tmp_path: Path, trajectory: dict | None, *, variant: str = "with")
tests/reporting/test_activation_validation.py:45
↓ 15 callersFunction_complete_current_run
(run_dir: Path)
tests/test_tier3_compare.py:26
↓ 15 callersFunction_fingerprint
(metadata: os.stat_result)
src/skillevaluator/tier3/harbor/secure_copy.py:217
↓ 15 callersFunction_model_for_agent
( agent: str, *, cli_model: str | None, config_agents: dict[str, Any], provider: ProviderC
src/skillevaluator/tier3/harbor/runner.py:3614
↓ 15 callersFunction_package
(root: Path)
tests/tier3/test_plugin_prepare_review.py:45
↓ 15 callersFunction_payload
(agents: dict[str, dict])
tests/reporting/test_dimension_explanations.py:50
↓ 15 callersFunction_plain_text
Strip ANSI escape codes (CI may set FORCE_COLOR, making rich emit them).
tests/test_commands.py:454
↓ 15 callersFunction_prepare
(root: Path)
tests/tier3/test_plugin_runtime_identity.py:41
↓ 15 callersFunction_project
()
tests/test_oss_packaging.py:85
↓ 15 callersFunction_publication_ready_results_with_conditions
Return publication-ready evidence with both required conditions.
tests/reporting/test_advisory_skip_verdict.py:251
↓ 15 callersFunction_render
(payload: dict[str, Any])
tests/reporting/test_tier3_baseline_trial_views.py:167
↓ 15 callersFunction_render
(result=None, **kwargs)
tests/reporting/test_native_feature_restoration.py:99
↓ 15 callersFunction_reserve_run_dir
Atomically reserve and authenticate a unique run directory.
src/skillevaluator/tier3/harbor/runner.py:950
↓ 15 callersFunction_results
One skill: the same secret from two scanners, mixed severities, a failing quality score.
tests/reporting/test_report_redesign.py:30
↓ 15 callersFunction_run_engine
Run ``run_harbor_eval`` against a real config file with the Harbor edges mocked.
tests/test_tier3_config_key_parity.py:48
↓ 15 callersFunction_saved_catalog
(tmp_path: Path, scan_root: Path, *, full_body: bool = False)
tests/embedding/test_catalog.py:643
↓ 15 callersFunction_source
(root: Path)
tests/tier3/test_plugin_runtime_identity.py:28
↓ 15 callersFunction_step
(call_id: str, fn: str, args: dict[str, Any], content: str = "ok", **extra: Any)
tests/tier3/test_f90_signals_handoff.py:37
↓ 15 callersFunction_target
(skill_dir: Path)
tests/embedding/test_catalog.py:659
↓ 15 callersFunction_text
(rows)
tests/reporting/test_tier3_validate_cases.py:38
↓ 15 callersFunction_tier3_display_identifier
(value: object)
src/skillevaluator/reporting/html.py:440
↓ 15 callersFunction_validate_with
(tmp_path: Path, *replies: str | Exception)
tests/validators/test_rubric_eval.py:1266
↓ 15 callersFunction_write_completed_run
(root: Path, run_id: str)
tests/test_results_location.py:22
↓ 15 callersMethodadmit
(self, *, agent: str, arm: str, case_id: str, attempt: int)
src/skillevaluator/tier3/harbor/run_budget.py:165
↓ 15 callersFunctionatif_content_text
Return the text of an ATIF message or observation content value. ATIF content is a string or a list of content parts. Text parts are joined;
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:24
↓ 15 callersFunctionbuild_behavior_evidence
Build compact, behavior-check-specific evidence from an ATIF trajectory. Behavior checks often ask whether the agent produced or changed artifact
src/skillevaluator/tier3/eval_core/atif_helpers.py:491
↓ 15 callersMethodcancel
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 15 callersFunctioncaptured_report
()
tests/validators/test_security_reference_caveats.py:42
↓ 15 callersFunctionclassify
Map a cosine similarity score to a classification tier and severity. The four fixed tiers are checked in descending order; the caller's --thr
src/skillevaluator/embedding/registry.py:132
↓ 15 callersFunctioncontext
(results, **kwargs)
tests/reporting/test_final_render_context.py:19
↓ 15 callersMethodextract_json
(content: str)
tests/validators/test_rubric_eval.py:1440
↓ 15 callersFunctionfinite_number
Return *value* as a finite float, or ``None`` when it is not a usable number. Booleans, non-numbers, NaN, infinities and integers too large for a
src/skillevaluator/tier3/harbor/metrics.py:276
↓ 15 callersMethodgetParsed
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 15 callersFunctionki
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 15 callersFunctionload_profiles
Load profiles by name (comma list or ``all``); unknown names raise ``ValueError``.
src/skillevaluator/harness/overlay.py:124
↓ 15 callersFunctionmanifest
()
tests/ci/test_tier_manifest.py:12
↓ 15 callersFunctionmodule
()
tests/tier3/test_private_runtime_evidence.py:12
↓ 15 callersFunctionnative_attempt
(root: Path, attempt: int, *, timeout: bool = False, reward: float = 1.0)
tests/test_harbor_timeout_recovery.py:89
↓ 15 callersFunctionparse_markdown
Split optional ``---`` YAML frontmatter from a Markdown body (bounded YAML). Frontmatter that strict YAML rejects is read again the way Claude Co
src/skillevaluator/plugin_markdown.py:87
↓ 15 callersFunctionpolicy
(**changes)
tests/tier3/test_grader_policy.py:17
↓ 15 callersFunctionprepare
(directory, **changes)
tests/ci/test_comment_report.py:20
↓ 15 callersFunctionredact
Redact string leaves, counting introduced secret markers and PII matches.
src/skillevaluator/tier3/trace_import.py:76
↓ 15 callersMethodrelease
Finish an otherwise unrecorded reservation with unknown usage, at most once.
src/skillevaluator/tier3/harbor/run_budget.py:261
↓ 15 callersFunctionrender_retained_report
Render a single skill from retained evidence without inspecting its paths. *scan_root* is the folder the scanners ran on. It is only compared wit
src/skillevaluator/reporting/retained.py:388
↓ 15 callersFunctionreplace
( match: re.Match[str], category: str = category, exceptions:
src/skillevaluator/tier3/trace_import.py:106
↓ 15 callersFunctionreport
()
tests/ci/test_publish_reports.py:76
↓ 15 callersMethodreport
(self, severity: Severity, check_name: str, message: str, suggestion: str)
src/skillevaluator/validators/mcp_static.py:405
↓ 15 callersFunctionresolve_embedding_provider
Resolve the embedding provider used by Tier 2 semantic overlap checks.
src/skillevaluator/provider_config.py:620
↓ 15 callersFunctionreward
(case, value=0.5, *, accuracy=None, judge=None)
tests/tier3/test_private_comparison_contracts.py:14
↓ 15 callersFunctionstage_for_gym
Adapt Harbor-format tasks for Gym's agents and verifier. Drops Harbor's ``skills_dir`` task hook, which Gym's CLI agents reject. The task image
src/skillevaluator/tier3/nemo_gym/runner.py:68
↓ 15 callersMethodstage_with_arm
(self, agent: str, task_dirs: Sequence[Path])
src/skillevaluator/tier3/plugin_execution.py:136
↓ 15 callersFunctiontarget
(er, url="https://public.example/path?token=secret#frag", **kwargs)
tests/validators/test_private_endpoint_resolution.py:27
↓ 15 callersMethodwait
(self)
tests/test_harbor_secure_docker_environment.py:3954
↓ 14 callersFunction_agent
Return loaded agent data in the shape ``report_data.load_agent_data`` produces.
tests/reporting/test_tier3_incomplete_arms.py:60
↓ 14 callersFunction_agent
(*, activated: bool = False, score: float = 0.4)
tests/reporting/test_skill_activation_reporting.py:29
↓ 14 callersFunction_arm
(trials: list[TrialObservation], status: str = "succeeded", job_failure: str = "")
tests/tier3/test_private_ladder_stats.py:59
↓ 14 callersFunction_attempts
(verifier: ModuleType)
tests/tier3/test_ragas_judge_evidence.py:249
↓ 14 callersFunction_bind_target
Bind one deterministic fixture result to the shared source snapshot.
tests/golden/test_benchmark_card.py:39
↓ 14 callersFunction_client
(dimension: int = 3, duplicate_last: bool = False)
tests/embedding/test_similarity_limits.py:30
↓ 14 callersFunction_codex_output
(exit_code: int, output: str = "")
tests/tier3/test_error_recovery.py:36
↓ 14 callersFunction_discover_trajectories
Find agent trajectories from existing eval results. Checks --from-results path first, then resolved latest results. Returns dict mapping case
src/skillevaluator/tier3/generate_dataset.py:1070
↓ 14 callersFunction_expected
(**overrides: str | None)
tests/test_benchmark_source_provenance.py:526
↓ 14 callersFunction_extract_json_reply
(monkeypatch: pytest.MonkeyPatch, content: str)
tests/inference/test_llm_client.py:800
↓ 14 callersFunction_failed_judge_artifacts
(entry_id: str, *, reason: str = "judge request returned HTTP 401")
tests/test_issue55_collector_truth.py:107
↓ 14 callersFunction_install
(monkeypatch: pytest.MonkeyPatch, **entries: str)
tests/test_custom_execution_backends.py:76
↓ 14 callersFunction_judge_evidence
Collect judge evidence the way main() does while it runs the judges.
tests/tier3/test_ragas_judge_evidence.py:240
↓ 14 callersFunction_load_json
(path: Path, *, root: Path | None = None)
src/skillevaluator/tier3/case_summary.py:882
↓ 14 callersFunction_logical_attempt_rewards
Collapse multi-step reward rows to one metric-bearing logical trial.
src/skillevaluator/tier3/harbor/collector.py:4725
↓ 14 callersFunction_marks
(case: Mapping[str, Any] | None)
src/skillevaluator/tier3/case_summary.py:359
↓ 14 callersFunction_openai_choice_response
(content: str | None, *, finish_reason: str = "stop", refusal: str | None = None)
tests/inference/test_llm_client.py:998
↓ 14 callersFunction_openclaw
(root: Path, files: dict[str, Any] | None = None, **fields: Any)
tests/deduplication/plugin/test_openclaw_profile.py:71
↓ 14 callersFunction_path_is_excluded
Check lexical and resolved containment using cross-platform path casing.
src/skillevaluator/tier3/harbor/adapter.py:531
↓ 14 callersFunction_payload
()
tests/reporting/test_telemetry_view.py:41
↓ 14 callersFunction_payload
( rewards: list[dict], expected_skill: object = "demo", *, baseline: list[dict] | None = None,
tests/reporting/test_activation_validation.py:78
↓ 14 callersFunction_render
(view)
tests/reporting/test_telemetry_view.py:20
↓ 14 callersFunction_rows
(**options)
tests/test_doctor.py:45
↓ 14 callersMethod_run_trusted_transfer_command
Stream untrusted output to disk with size, idle, and total bounds.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:2525
↓ 14 callersFunction_skill_dir
(tmp_path: Path, *, guidance: str | None = None)
tests/tier3/test_dataset_grounding.py:19
↓ 14 callersFunction_skillspector_child_env
Return the SkillSpector environment plus SkillEvaluator's default reasoning effort.
src/skillevaluator/validators/security.py:778
← previousnext →501–600 of 24,741, ranked by callers