MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 24 callersFunctionwrite_decoy_manifest
Atomically save a bounded JSON selection without following output links.
src/skillevaluator/tier3/decoys.py:353
↓ 23 callersFunction_bind_publication_target
Bind fixture results and duplicated Tier 3 claims to one target.
tests/reporting/test_advisory_skip_verdict.py:72
↓ 23 callersFunction_trace
(skill, results_root, case_id="sample-skill-001", *, steps=None, skill_name=None)
tests/tier3/test_dataset_auto_grounding.py:27
↓ 23 callersFunction_write_actual_harbor_022_result
Persist a real Harbor 0.22 JobResult and its TrialResult artifact.
tests/test_harbor_collector_runtime_failures.py:55
↓ 23 callersFunctionchunk_file
Route to the appropriate chunker based on file extension.
src/skillevaluator/deduplication/utils/chunker.py:55
↓ 23 callersMethodclose
(self)
src/skillevaluator/utils/structured_data.py:368
↓ 23 callersMethodclose
(self)
tests/test_model_catalog.py:368
↓ 23 callersFunctioncompletion
(content, **choice)
tests/tier3/test_judge_policies.py:82
↓ 23 callersFunctionget_logger
Get a configured logger instance. Args: name: Logger name (typically __name__) level: Logging level (default: INFO) Returns:
src/skillevaluator/logging_config.py:50
↓ 23 callersFunctioninventory
(root, *, skill_paths=("custom/a",), name="bundle", description="A useful bundle", complete=True)
tests/deduplication/plugin/test_private_dedup.py:59
↓ 23 callersFunctioniter_scannable_files
Walk ``root`` and return files matching any of ``extensions``. Skips files whose path traverses any directory listed in ``excluded_dirs`` (de
src/skillevaluator/validators/base.py:73
↓ 23 callersFunctionload_decoy_manifest
Verify a saved selection against local sources, without model access. Paths and content hashes are checked, not silently refreshed. Callers that
src/skillevaluator/tier3/decoys.py:363
↓ 23 callersFunctionpublication_semantic_text
Normalize public text and remove invisible identity-spoofing characters.
src/skillevaluator/publication_text.py:341
↓ 23 callersMethodrender_all
( self, results: list[ValidationResult], *, plugin_static: object = None,
src/skillevaluator/reporting/html.py:3451
↓ 23 callersFunctionwrite_tree
(root: Path, files: dict)
tests/test_plugin_discovery.py:31
↓ 22 callersFunction_assert_counts_match
(root: ET.Element)
tests/reporting/test_junit_reporter.py:31
↓ 22 callersFunction_call
(action: str, **args)
tests/tier3/test_tool_productivity.py:25
↓ 22 callersFunction_complete_checks
( *, scores: dict[str, int | float] | None = None, passes: dict[str, bool] | None = None, )
tests/validators/test_rubric_eval.py:47
↓ 22 callersFunction_html_report_data
(output: str)
tests/reporting/test_reporters.py:68
↓ 22 callersFunction_publication_target
Return a deterministic valid source identity for reporter fixtures.
tests/reporting/test_advisory_skip_verdict.py:62
↓ 22 callersFunction_read_json
Read one bounded regular JSON file through an anchored no-follow root.
src/skillevaluator/tier3/harbor/collector.py:514
↓ 22 callersFunction_reward
( entry_id: str = "case-1", trial_id: str = "case-1__attempt1", *, security: float = 0.0,
tests/reporting/test_tier3_failure_reasons.py:69
↓ 22 callersFunction_skill
(**overrides)
tests/tier3/test_dataset_baseline_fairness.py:390
↓ 22 callersFunction_user
(second: int = 0)
tests/tier3/test_trajectory_timing.py:43
↓ 22 callersFunction_write_anchor_allowlist
( path: Path, *, allow_path: str = "tests/test_negative.py", rule: str = "internal-nvidia-cred
tests/test_oss_boundary.py:155
↓ 22 callersFunction_write_findings
(result: dict)
tests/tier3/test_apply_patch_security.py:71
↓ 22 callersFunction_write_reward
( jobs_dir: Path, *, variant: str, case_id: str, attempt: int, score: float = 0.25,
tests/test_harbor_collector_runtime_failures.py:1779
↓ 22 callersFunctionagent_eval_publication_evidence_complete
Return whether a payload carries the minimum publication Tier 3 evidence.
src/skillevaluator/reporting/base.py:1501
↓ 22 callersFunctionbudget_line
(metadata: dict[str, Any], *, agents: dict[str, Any] | None = None)
src/skillevaluator/tier3/harbor/run_budget.py:497
↓ 22 callersMethodclose
(self)
tests/tier3/test_skip_with_skill.py:232
↓ 22 callersFunctiond
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 22 callersMethodfail_gate
Record a failed requirement that severity overrides cannot waive.
src/skillevaluator/models/result.py:679
↓ 22 callersFunctionfound
(root: Path)
tests/test_plugin_discovery.py:42
↓ 22 callersMethodisHorizontal
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 22 callersFunctionnormalized_evaluated_source
Return the validated evaluated-source identity, or ``None``. Each field is accepted only in its canonical shape, so a published card can neve
src/skillevaluator/source_identity.py:115
↓ 22 callersMethodrun
(self, instruction: str, environment: BaseEnvironment, context: AgentContext)
src/skillevaluator/tier3/harbor/hermes_agent.py:67
↓ 22 callersMethodset
(self)
src/skillevaluator/tier3/harbor/runner.py:918
↓ 21 callersFunction_collect_supplementary_content
Collect complete linked Markdown before optional bounded previews.
src/skillevaluator/validators/rubric_eval.py:484
↓ 21 callersMethod_convert_skillspector_issue
Convert a single skillspector issue dict into a Finding. Returns (Finding, is_error) where is_error is True for CRITICAL/HIGH. An ``
src/skillevaluator/validators/security.py:5669
↓ 21 callersFunction_load
(name: str)
tests/test_ci_workflows.py:62
↓ 21 callersFunction_make_skill
Create a single file inside a skill-like directory.
tests/validators/test_unicode_smuggle.py:18
↓ 21 callersFunction_native_run
(path: Path)
tests/reporting/test_dashboard_data.py:675
↓ 21 callersFunction_payload
(rewards: list[dict[str, Any]], baseline: list[dict[str, Any]] | None = None)
tests/reporting/test_tier3_failure_reasons.py:93
↓ 21 callersFunction_render
(payload: dict[str, Any])
tests/reporting/test_tier3_failure_reasons.py:116
↓ 21 callersMethod_run
( self, command: list[str], run_root: Path, *, allow_net: bool = False
tests/test_harbor_local_sandbox.py:1075
↓ 21 callersFunction_security
(copy: str, tool_call: dict[str, Any], *, user_text: str = "", agent_text: str = "Done.")
tests/tier3/test_security_finding_evidence.py:311
↓ 21 callersFunction_subtab
(page, name: str)
tests/reporting/test_report_browser.py:481
↓ 21 callersFunction_waiver
(**overrides: object)
tests/validators/test_pii_waivers.py:30
↓ 21 callersFunctionarchive
(files)
tests/ci/test_publish_reports.py:39
↓ 21 callersFunctionbundle
()
tests/reporting/test_private_static_reports.py:43
↓ 21 callersMethodcontent_report_for_tasks
The :meth:`content_report` of tasks the runner emitted: ``{arm: [task dir, ...]}`` per compared arm. Each task's ``environment/`` is hashed a
src/skillevaluator/tier3/loose_parts.py:411
↓ 21 callersFunctionemit_reports
Render reports and return whether every result passed. When *policy* is provided, finding severities are remapped per the active profile (and
src/skillevaluator/tier1/commands.py:643
↓ 21 callersFunctionextract_custom_metrics
Return user/custom numeric metrics that are separate from SkillEvaluator metrics.
src/skillevaluator/tier3/harbor/metrics.py:751
↓ 21 callersMethodinstall
(self, environment: BaseEnvironment)
src/skillevaluator/tier3/harbor/hermes_agent.py:49
↓ 21 callersFunctionmake_cluster
Factory for ContentCluster objects.
tests/deduplication/conftest.py:76
↓ 21 callersFunctionnative_source
(plugin: Path, stage_root: Path)
tests/tier3/private_loose_fixtures.py:142
↓ 21 callersMethodreset
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 21 callersFunctionsample
()
tests/reporting/test_private_runtime_projection.py:17
↓ 21 callersMethodsave
Save rendered output to file. Args: results: List of ValidationResults to render output_path: Path to save the output
src/skillevaluator/reporting/base.py:2697
↓ 21 callersFunctionselect
(repository, base, head, roots="skills")
tests/ci/test_select_changed_skills.py:51
↓ 21 callersMethodsleep
(self, seconds: float)
src/skillevaluator/tier3/mcp_proof.py:476
↓ 21 callersMethodstart
(self, plan: Any)
tests/tier3/test_skip_with_skill.py:220
↓ 20 callersFunction_collect
(tmp_path: Path, *, skip_baseline: bool = True, expected_trials: int = 1)
tests/test_harbor_negative_control_evidence.py:105
↓ 20 callersFunction_failed_payload
()
tests/reporting/test_insights_view.py:18
↓ 20 callersFunction_materialize_trajectory_file
( agent_dir: Path, reference: Any, *, state: _TrajectoryReferenceState | None = None, dept
src/skillevaluator/tier3/harbor/collector.py:2758
↓ 20 callersFunction_render
(results: list[ValidationResult])
tests/reporting/test_report_redesign.py:56
↓ 20 callersFunction_reply
Serialize a judge reply whose own totals disagree with its criterion scores.
tests/validators/test_rubric_eval.py:1221
↓ 20 callersFunction_tier1_result
()
tests/test_public_benchmark_gate.py:830
↓ 20 callersFunction_tier2_result
()
tests/test_public_benchmark_gate.py:837
↓ 20 callersFunction_write_harbor_0132_unscored_result
(jobs_dir: Path)
tests/test_harbor_runtime_preflight.py:98
↓ 20 callersFunction_write_job_result
(job_dir: Path, trial_names: list[str])
tests/test_harbor_negative_control_evidence.py:62
↓ 20 callersFunction_write_waivers
(root: Path, *waivers: dict)
tests/validators/test_pii_waivers.py:57
↓ 20 callersFunctionanalyze_cluster
Run LLM analysis on a single content cluster. The reply is checked before it can become a finding: a verdict outside ``VALID_VERDICTS``, a co
src/skillevaluator/deduplication/intra_skill/llm_analyzer.py:199
↓ 20 callersFunctionapi
()
tests/reporting/test_native_evidence.py:14
↓ 20 callersFunctionattempt_verdict
Apply required checks before the existing Overall threshold.
src/skillevaluator/tier3/attempt_verdict.py:139
↓ 20 callersFunctionbundle
(root: Path)
tests/validators/test_private_plugin_composition.py:13
↓ 20 callersMethodcancel
(self)
src/skillevaluator/tier3/harbor/run_budget.py:256
↓ 20 callersFunctioncheck_routing
Check the agent read only expected/allowed workspace skill docs.
src/skillevaluator/tier3/eval_core/checks.py:7782
↓ 20 callersFunctioncompare_results
Compare latest Harbor result scores across agents.
src/skillevaluator/tier3/commands.py:1469
↓ 20 callersMethodkind
(self)
src/skillevaluator/plugin_mcp.py:90
↓ 20 callersFunctionload_profile
Load a bundled policy (``default`` or ``registry``). ``external`` is a compatibility alias for ``default``. Raises: FileNotFoundErro
src/skillevaluator/validators/policy.py:262
↓ 20 callersFunctionnext_steps
Return up to three distinct run-level next steps, most blocking first.
src/skillevaluator/tier3/next_steps.py:126
↓ 20 callersMethodparse_response
(self, _response_text: str, **_kwargs)
tests/inference/test_llm_client.py:1819
↓ 20 callersFunctionredact_progress_detail
Return a single-line diagnostic safe enough for a progress surface.
src/skillevaluator/tier3/harbor/progress.py:150
↓ 20 callersFunctionrender
(payload)
tests/ci/test_comment_summary.py:49
↓ 20 callersFunctionretain
(reports, *, names=("demo",), tiers=(1, 2, 3), verdict="pass")
tests/ci/test_benchmark_reports.py:92
↓ 20 callersFunctionrun_plugin_validation
Assemble a private run; network collectors require independent explicit options.
src/skillevaluator/tier1/plugin_checks.py:260
↓ 20 callersFunctionsource
(tmp_path, files)
tests/validators/test_private_dependency_audit.py:47
↓ 20 callersMethodwrite
(self, buffer, timeout=None)
tests/tier3/test_mcp_proof_sdk_compat.py:291
↓ 19 callersFunction_agent
(second: int)
tests/tier3/test_trajectory_timing.py:47
↓ 19 callersFunction_cases
(agent: Mapping[str, Any], key: str)
src/skillevaluator/tier3/case_summary.py:180
↓ 19 callersFunction_cases
(root: ET.Element)
tests/reporting/test_junit_reporter.py:23
↓ 19 callersFunction_collision_safe_redaction_marker
Build a marker that cannot contain or join into an eligible secret.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:285
↓ 19 callersFunction_ids
(rows: dict[str, dict], harness: str)
tests/harness/test_overlay.py:339
↓ 19 callersFunction_patch
(header: str)
tests/tier3/test_apply_patch_security.py:37
↓ 19 callersFunction_path_is_link_or_reparse
Return whether a path is a symlink, junction, or Windows reparse point.
src/skillevaluator/tier3/harbor/adapter.py:552
↓ 19 callersFunction_persisted_reward
(tmp_path: Path, variant: str, trial_name: str)
tests/test_harbor_negative_control_evidence.py:117
↓ 19 callersFunction_reward
(metric_score=0.1)
tests/tier3/test_suggestion_grounding.py:13
↓ 19 callersFunction_run
( root: Path, hour: int, *, score=0.8, model="model-a", policy="policy-a", revisio
tests/test_tier3_history.py:27
↓ 19 callersMethod_run_skillspector
Execute skillspector security scan via CLI on a single directory. Runs `skillspector scan <path> --format json`; adds `--no-llm` when
src/skillevaluator/validators/security.py:3301
← previousnext →301–400 of 24,741, ranked by callers