MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 13 callersMethodupload_dir
(self, source_dir: Path | str, target_dir: str)
tests/test_harbor_claude_refusal_fallback.py:577
↓ 13 callersFunctionurl_credentials
Credentials written into the text of ``url``: in its userinfo, query parameters, and fragment parameters. MCP server URLs, HTTP hook URLs, and UR
src/skillevaluator/validators/url_policy.py:429
↓ 13 callersMethodvalidate
Run similarity detection on a path. Without a catalog, performs pairwise comparison of discovered items. With a catalog, compares exa
src/skillevaluator/validators/similarity.py:84
↓ 13 callersFunctionverdict
Return (label, per-agent detail) using the raw per-attempt success rate.
src/skillevaluator/tier3/case_summary.py:398
↓ 12 callersFunction_assert_publication_incomplete_across_reporters
Assert imported evidence fails closed consistently in every report view.
tests/reporting/test_advisory_skip_verdict.py:262
↓ 12 callersFunction_card
(evaluated_source: dict[str, str] | None)
tests/test_benchmark_source_provenance.py:55
↓ 12 callersFunction_card
(agent: dict, metric: str = "accuracy")
tests/test_judge_lift_integrity.py:350
↓ 12 callersMethod_check_item_count
Expose every bounded-list cut before declarations or directory entries are dropped.
src/skillevaluator/plugin_discovery.py:294
↓ 12 callersFunction_collection
(root: Path, malformed: int = 1)
tests/embedding/test_collection_coverage.py:21
↓ 12 callersFunction_command_basename
(command: str)
src/skillevaluator/validators/mcp_static.py:1686
↓ 12 callersFunction_commit
(repo: Path, message: str)
tests/test_ci_change_classifier.py:30
↓ 12 callersFunction_compute_lift
Compute skill lift (with-skill minus without-skill) per metric. A metric that an arm recorded as not applicable in every trial is absent from
src/skillevaluator/tier3/harbor/collector.py:5600
↓ 12 callersFunction_dict
(value: Any)
src/skillevaluator/reporting/dashboard_data.py:61
↓ 12 callersFunction_environment
(tmp_path: Path)
tests/test_harbor_local_upload.py:23
↓ 12 callersFunction_evidence_block
(rendered: str)
tests/reporting/test_verdict_evidence.py:104
↓ 12 callersFunction_extract_path
Extract a file path argument from a tool call, handling multiple field names.
src/skillevaluator/tier3/eval_core/checks.py:866
↓ 12 callersFunction_finite_number
Return a bounded finite number, rejecting booleans and shaped values.
src/skillevaluator/reporting/html.py:186
↓ 12 callersFunction_fixture_value
()
tests/ci/test_secret_redaction.py:31
↓ 12 callersFunction_issue_192_results
Build the run issue #192 reports, as current ``main`` can produce it. ``claude-code`` is the agent whose dimension scores were withheld, so the
tests/reporting/test_verdict_evidence.py:51
↓ 12 callersFunction_launch
(call_id: str, name: str, **extra: Any)
tests/tier3/test_f90_signals_handoff.py:60
↓ 12 callersFunction_load_mcp_servers
Load MCP server declarations from evals/environment/mcp_servers.toml.
src/skillevaluator/tier3/harbor/adapter.py:2142
↓ 12 callersFunction_missing_reference_warning
(labels: str, count: int = 1)
tests/validators/test_security.py:9354
↓ 12 callersFunction_payload
(*, include_baseline: bool = True)
tests/reporting/test_tier3_baseline_trial_views.py:141
↓ 12 callersFunction_plain
Drop HTML tags, entities and Markdown emphasis, and collapse whitespace.
tests/reporting/test_tier3_completed_agent_docs.py:33
↓ 12 callersFunction_provenance
()
tests/test_skillspector_provenance.py:28
↓ 12 callersFunction_published_trial_label
Return a bounded value-only label; unsafe identities never become keys or paths.
src/skillevaluator/tier3/harbor/collector.py:370
↓ 12 callersFunction_render
(payload: dict)
tests/reporting/test_dimension_explanations.py:56
↓ 12 callersFunction_render_report
( skill_name: str, results_dir: Path, output_path: Path | None = None, skill_path: Path | None
tests/test_harbor_html_report_failures.py:24
↓ 12 callersFunction_result
(state="complete")
tests/reporting/test_report_clarity.py:15
↓ 12 callersFunction_reward
All six default metrics at *value*; a keyword sets one metric (``None`` = not applicable).
tests/tier3/test_private_ladder_stats.py:40
↓ 12 callersFunction_safe_diagnostic_text
Return redacted, bounded text without terminal-control characters.
src/skillevaluator/tier3/harbor/collector.py:338
↓ 12 callersFunction_similarity_result
(*, severity: Severity = Severity.MEDIUM)
tests/reporting/test_reporters.py:101
↓ 12 callersFunction_skill
(name: str)
tests/tier3/test_plugin_prepare_review.py:35
↓ 12 callersFunction_spec
(**overrides: object)
tests/test_custom_execution_backends.py:44
↓ 12 callersFunction_tier3_result
( *, environment: str, metric_label: str = "Accuracy", skill_name: str = "demo-skill", )
tests/golden/test_benchmark_card.py:106
↓ 12 callersFunction_unsafe_report_path
(path: Path, reason: str)
src/skillevaluator/reporting/base.py:127
↓ 12 callersFunction_unverified
(result: ValidationResult)
tests/validators/test_skillspector_hook_surface.py:334
↓ 12 callersFunction_write
(path: Path, value: object)
tests/test_baseline_store.py:23
↓ 12 callersFunction_write_authenticated_current_run
( root: Path, run_id: str, *, skill_name: str = "demo", recorded_run_dir: str | None = Non
tests/test_results_location.py:33
↓ 12 callersFunction_write_config
(tmp_path: Path, body: str)
tests/test_quality_config.py:291
↓ 12 callersFunction_write_fixture
(tmp_path: Path, failed=False)
tests/ci/test_native_reports.py:21
↓ 12 callersFunction_write_skill
( tmp_path: Path, description: str, eval_md: str | None = None, scripts: tuple[str, ...] = ("review.py",)
tests/tier3/test_dataset_baseline_fairness.py:1256
↓ 12 callersFunction_write_skill_manifest
(root: Path, name: str, manifest: str)
tests/embedding/test_extractor.py:713
↓ 12 callersMethodadd_finding
(self, finding)
tests/validators/test_plugin_dependencies.py:327
↓ 12 callersFunctionapi
()
tests/reporting/test_plugin_evidence_report.py:17
↓ 12 callersFunctionapply_in_agent_mcp_proof
Update each server's status from the with-plugin arm's MCP calls and load census. * At least one successful call: ``used-successfully``, whatever
src/skillevaluator/tier3/mcp_proof.py:904
↓ 12 callersFunctionassess_dataset_sufficiency
Assess dataset coverage, pass-rate reliability, and paired lift coverage.
src/skillevaluator/tier3/dataset_sufficiency.py:62
↓ 12 callersFunctionbuild_candidates
Return only redacted review metadata and fixture contents, without writing.
src/skillevaluator/tier3/trace_import.py:243
↓ 12 callersFunctioncases
()
tests/tier3/test_grader_policy.py:28
↓ 12 callersMethodclose
(self)
src/skillevaluator/tier3/harbor/progress.py:275
↓ 12 callersFunctioncollect
(root: Path, *attempt_jobs: Path, stop_on_pass: bool = True)
tests/test_harbor_timeout_recovery.py:138
↓ 12 callersFunctioncompleted_agent_result
Return a display-only summary of the canonical best completed agent. This never changes the payload, execution status, or publication assessment.
src/skillevaluator/reporting/tier3_scope.py:18
↓ 12 callersMethodcontent_report
``content_hashes`` of the full, loose, and skills-only arms, ``equal_across_arms`` and ``mismatches``.
src/skillevaluator/tier3/loose_parts.py:399
↓ 12 callersMethodcreate
(self, branch, base, files, title, body, *, setup=False)
src/skillevaluator/_ci_onboarding_all_tiers.py:298
↓ 12 callersFunctiondata_rows
(body)
tests/ci/test_comment_summary.py:55
↓ 12 callersFunctiondetect_content_type
Auto-detect whether path contains a skill, rules, workflows, or plugin. Detection order: file type -> directory manifests -> path patterns -> nes
src/skillevaluator/cli_core.py:98
↓ 12 callersFunctiondiscover_plugin_members
Locate and read actual plugin declarations without activating public validators. Each supported co-located manifest gets its own profile and prov
src/skillevaluator/plugin_discovery.py:1527
↓ 12 callersMethodgetDataset
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 12 callersMethodgetLabels
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 12 callersFunctionlooks_like_inline_secret
True when a keyed value is an inline credential rather than a reference. A value shaped like a secret counts under any key, including an inline
src/skillevaluator/validators/url_policy.py:279
↓ 12 callersFunctionmatches_filesystem_name
Match an entry name according to the host filesystem's case semantics. Exact spellings retain the caller's historical behavior. Differently c
src/skillevaluator/utils/path_security.py:48
↓ 12 callersFunctionmodel_environment
Forward selected provider settings, never ambient CI or vendor credentials.
ci/model_config.py:83
↓ 12 callersFunctionmonotonic
()
tests/ci/test_run_tier1.py:1551
↓ 12 callersFunctionpasses_required_gate
Return whether *result* permits the required validation gate to pass.
src/skillevaluator/reporting/base.py:2537
↓ 12 callersFunctionpreflight_json_structure
Lexically bound JSON before ``json.loads`` materializes nested pairs. Tokens are strings, opened arrays and objects, and item separators. A s
src/skillevaluator/utils/structured_data.py:376
↓ 12 callersFunctionquality_config_digest
Return a stable digest of the normalized rules, independent of file formatting.
src/skillevaluator/quality_config.py:418
↓ 12 callersFunctionreplace
(target: Path, handle, content: bytes, previous)
src/skillevaluator/ci_onboarding.py:357
↓ 12 callersFunctionresolve_plugin_load
Decide native or wrapper loading per agent. ``wrapper`` keeps today's behavior. ``auto`` uses the native adapter when it supports the agent a
src/skillevaluator/tier3/plugin_native.py:2231
↓ 12 callersFunctionresult_with_completed_peer
(*, baseline=0.4, peer_failed=True)
tests/reporting/test_tier3_scoped_reports.py:19
↓ 12 callersFunctionrun_interaction
Freeze reviewed inputs once, then delegate every arm to the Tier 3 service.
src/skillevaluator/tier3/interactions.py:288
↓ 12 callersFunctionscan
(tmp_path: Path, frontmatter: str, body: str = "", name: str = "SKILL.md")
tests/validators/test_pii_ownership.py:20
↓ 12 callersFunctionscore_policy_for_metrics
Return the versioned overall-score policy for one metric set.
src/skillevaluator/tier3/harbor/metrics.py:539
↓ 12 callersFunctionsections
(payload)
tests/reporting/test_private_runtime_projection.py:102
↓ 12 callersMethodseverity_for
Return severity for ``CATEGORY.check_name`` or ``CATEGORY.*``.
src/skillevaluator/validators/policy.py:102
↓ 12 callersFunctionsource
(root)
tests/tier3/test_private_comparison_staging.py:13
↓ 12 callersFunctionspec
(security, root="/workspace")
tests/tier3/test_private_plugin_security.py:38
↓ 12 callersMethodtext
(self, text: str)
src/skillevaluator/utils/secret_redaction.py:149
↓ 12 callersMethodvalidate
Run context deduplication on a single skill directory.
src/skillevaluator/deduplication/intra_skill/intra_skill_validator.py:200
↓ 12 callersFunctionvalidate_output_directory_path
Reject link/reparse components before creating an evaluator output tree.
src/skillevaluator/tier3/case_ids.py:136
↓ 12 callersFunctionvisible_markdown
Assert what a GitHub-style Markdown reader sees, including escaped punctuation.
tests/ci/test_report_views.py:87
↓ 11 callersFunction$
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 11 callersFunction_arguments
(tmp_path: Path)
tests/tier3/test_decoy_commands.py:43
↓ 11 callersFunction_case
(**fields: object)
tests/tier3/test_dataset_migration.py:30
↓ 11 callersFunction_caveat_error
(labels: str, *, description: str = "caveats that SkillEvaluator does not accept")
tests/validators/test_security.py:11181
↓ 11 callersFunction_classify
( repo: Path, base: str, head: str, output: Path, monkeypatch: pytest.MonkeyPatch, )
tests/test_ci_change_classifier.py:186
↓ 11 callersFunction_client
()
tests/embedding/test_collection_coverage.py:32
↓ 11 callersFunction_copy_verifier
Copy verifier code into a clean evaluator-owned tests subdirectory.
src/skillevaluator/tier3/harbor/adapter.py:2434
↓ 11 callersFunction_dict_list
(value: object)
src/skillevaluator/reporting/html.py:436
↓ 11 callersFunction_drilldown_table
(html: str)
tests/reporting/test_tier3_baseline_trial_views.py:173
↓ 11 callersFunction_eligible_secret_values
( secret_values: Iterable[str], *, include_short: bool = False, )
src/skillevaluator/tier3/harbor/secure_docker_environment.py:248
↓ 11 callersFunction_evaluate
(monkeypatch: pytest.MonkeyPatch, tmp_path: Path, **overrides)
tests/test_nemo_gym_backend.py:386
↓ 11 callersFunction_extract_rewards
Extract reward.json from all trials in a job directory.
src/skillevaluator/tier3/harbor/collector.py:3648
↓ 11 callersFunction_finding
(severity: Severity, *, category: str = "TEST", check: str = "check")
tests/models/test_result.py:21
↓ 11 callersFunction_heredoc
(command: str, patch: str = _BASHRC_PATCH)
tests/tier3/test_apply_patch_security.py:44
↓ 11 callersFunction_hook_rows
(result: ValidationResult)
tests/validators/test_plugin_component_risk.py:60
↓ 11 callersFunction_inspect_trial_directory
Classify one job child with one lstat, without following links.
src/skillevaluator/tier3/harbor/collector.py:1124
↓ 11 callersFunction_is_link_or_reparse
(metadata: os.stat_result)
src/skillevaluator/tier3/output_provenance.py:34
↓ 11 callersFunction_load_verifier_template
()
tests/test_tier3_public_runtime.py:49
↓ 11 callersFunction_package
(root: Path)
tests/tier3/test_openclaw_prepare.py:46
↓ 11 callersFunction_plugin
(root: Path, files: dict[str, str | bytes | dict])
tests/validators/test_plugin_hook_risk_bounds.py:43
← previousnext →701–800 of 24,741, ranked by callers