MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 19 callersFunction_shell_facts
What shell ``text`` does; ``remote_code`` says whether it runs downloaded content. Covers a fetch piped into an interpreter that reads its progra
src/skillevaluator/plugin_component_risk.py:2100
↓ 19 callersFunction_skill
(catalog: Path, relative: str, *, valid: bool = True, filename: str = "SKILL.md")
tests/test_catalog_discovery.py:22
↓ 19 callersFunction_skill
(workspace: Path)
tests/ci/test_secret_redaction.py:36
↓ 19 callersFunction_value
(card: str, field: str)
tests/test_benchmark_source_provenance.py:70
↓ 19 callersFunction_write_summary
( run_dir: Path, *, status: str = "succeeded", errors: list[str] | None = None, score: flo
tests/reporting/test_unified_tier3_report.py:52
↓ 19 callersFunction_write_trial
( jobs_dir: Path, *, variant: str, trial_name: str, reward: dict[str, Any], trajectory
tests/test_harbor_negative_control_evidence.py:85
↓ 19 callersFunctioncall
(command, tool="Bash", **kwargs)
tests/tier3/test_private_plugin_security.py:30
↓ 19 callersMethodchild_environment
Return this provider's public credential settings for a child process.
src/skillevaluator/provider_config.py:233
↓ 19 callersFunctionlocate_plugin_manifest
Locate one regular single-link manifest beneath a regular plugin root. All supported manifest variants are selected during discovery, so a linked
src/skillevaluator/plugin_manifest.py:508
↓ 19 callersFunctionmake_plugin
Write the full ``shipit`` plugin under ``root/<name>`` and return its directory.
tests/tier3/private_loose_fixtures.py:67
↓ 19 callersFunctionprobe_mcp_server
Probe one URL MCP server; return ``{"status", "tools", "detail"}``. ``expand_env`` holds the private environment grants: ``NAME`` (any server
src/skillevaluator/tier3/mcp_proof.py:701
↓ 19 callersFunctiontoolset
(**overrides)
tests/validators/test_private_dependency_audit.py:55
↓ 19 callersFunctionverify_source_identity
Re-read disk identity before/after staging, even when the caller retains an old inventory. The analyzed publication digest includes authored eval
src/skillevaluator/tier3/plugin_prepare.py:256
↓ 19 callersFunctionwrite
(path: Path, text: str, *, mode: int | None = None)
tests/tier3/private_loose_fixtures.py:60
↓ 18 callersFunction_chat_reply
(content: str)
tests/tier3/test_ragas_judge_evidence.py:182
↓ 18 callersFunction_copytree_secure
(source: Path, destination: Path, **kwargs: object)
tests/test_harbor_secure_copy.py:31
↓ 18 callersFunction_generate
( tmp_path: Path, compose: dict[str, Any], *, runtime_env: dict[str, str] | None = None, )
tests/test_harbor_custom_compose_safety.py:32
↓ 18 callersFunction_inferred
(llm: float, tool: float)
tests/tier3/test_trajectory_timing.py:25
↓ 18 callersFunction_make_skill
(tmp_path: Path)
tests/test_harbor_input_staging.py:51
↓ 18 callersFunction_normalize_trials
Project raw Harbor reward dicts into canonical per-trial entries. Each reward (loaded by ``_load_agent_data``) carries the per-evaluator scor
src/skillevaluator/evaluation/tier3_report.py:3532
↓ 18 callersFunction_openclaw
(root: Path, files: dict[str, Any] | None = None, **fields: Any)
tests/test_openclaw_plugin_discovery.py:40
↓ 18 callersMethod_parse
(self, *messages: str, deterministic: dict | None = None, case_id: str = "positive-001")
tests/evaluation/test_judges.py:960
↓ 18 callersFunction_skill
(tmp_path: Path)
tests/test_harbor_scaffold_transactions.py:21
↓ 18 callersFunction_stage_task_inputs
( env_dir: Path, *, input_files_dir: Path | None, entry: dict[str, Any], source_skill_path
src/skillevaluator/tier3/harbor/adapter.py:4132
↓ 18 callersFunction_tool_step
(second: int, durations: list[float])
tests/tier3/test_trajectory_timing.py:29
↓ 18 callersFunction_trajectory
(*, skill: str | None = None)
tests/test_harbor_negative_control_evidence.py:37
↓ 18 callersFunction_write_skill
(tmp_path: Path, case_ids: list[object])
tests/test_harbor_case_id_safety.py:24
↓ 18 callersFunction_write_variant_job_results
Persist a complete Harbor job result covering every staged trial directory.
tests/test_harbor_collector_runtime_failures.py:1821
↓ 18 callersFunctionapi
()
tests/reporting/test_private_static_reports.py:24
↓ 18 callersFunctioncollect_history
Read the first populated results root, with the usual legacy fallback.
src/skillevaluator/tier3/history.py:302
↓ 18 callersFunctioncommit
(repo)
tests/ci/test_select_changed_skills.py:29
↓ 18 callersFunctiondiscover_secure_files
Discover selected files below ``root`` without following redirects. Excluded directories are pruned before they consume the path budget. Ever
src/skillevaluator/utils/secure_fs.py:1115
↓ 18 callersFunctionfragment
(view, slot)
tests/reporting/test_native_evidence.py:23
↓ 18 callersFunctionharbor_job_links
Project every normalized job and unmatched legacy link, without attribution.
src/skillevaluator/reporting/harbor_viewer.py:108
↓ 18 callersFunctioninvoke
(repository, release_file, *args, input_text=None)
tests/test_ci_onboarding.py:65
↓ 18 callersFunctionmanifest
()
tests/ci/test_comment_summary.py:13
↓ 18 callersFunctionprivate_evaluator_skill_snapshot
Yield one selective, immutable evaluator projection for a complete run.
src/skillevaluator/tier3/harbor/adapter.py:6664
↓ 18 callersFunctionpublication_identity_present
Return whether identity text records non-placeholder provenance.
src/skillevaluator/publication_text.py:405
↓ 18 callersFunctionresult_status
(result: ValidationResult)
src/skillevaluator/reporting/run_status.py:109
↓ 18 callersFunctionrun_precommit
Run *check* once per skill touched by *paths*. Returns ``(exit_code, lines)``. Exit code is 1 when any skill fails and 0 otherwise, including
src/skillevaluator/precommit.py:228
↓ 18 callersFunctionsecret_values_from_environment
Extract exact credential values without treating every env value as secret.
src/skillevaluator/tier3/harbor/progress.py:171
↓ 18 callersFunctionsession
(**kwargs)
tests/test_trace_import.py:33
↓ 18 callersFunctionsummarize_results
Return observed issue counts, completeness and one safe highest-severity issue. Call only after ``retained_results`` validates the native documen
ci/summary_metrics.py:63
↓ 18 callersMethodunion
(self, x: int, y: int)
src/skillevaluator/deduplication/intra_skill/semantic_clustering.py:50
↓ 18 callersMethodvalidate
Run license compliance validation on asset(s) at path.
src/skillevaluator/validators/license.py:241
↓ 18 callersMethodvalidate_security_only
Run only skillspector security scanning (no PII detection).
src/skillevaluator/validators/security.py:3188
↓ 17 callersFunctionSi
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 17 callersFunction_agent_result
(skill_name: str, agent_names: list[str])
tests/reporting/browser_navigation_fixtures.py:80
↓ 17 callersFunction_classify
Classify one call through check_tool_efficiency, which both copies expose.
tests/tier3/test_tool_productivity.py:29
↓ 17 callersFunction_finding_names
(result)
tests/validators/test_version.py:42
↓ 17 callersFunction_hooks
(events: dict)
tests/validators/test_plugin_component_risk.py:51
↓ 17 callersFunction_pre_tool_use
(*handlers: dict, matcher: str | None = None)
tests/validators/test_plugin_hook_risk_bounds.py:60
↓ 17 callersFunction_publication_safe_inline
Render untrusted metadata as one publication-safe Markdown line.
src/skillevaluator/reporting/benchmark.py:2219
↓ 17 callersFunction_shell_executable
Extract normalized executable name from command token.
src/skillevaluator/tier3/eval_core/script_invocation.py:94
↓ 17 callersFunction_skill
(root: Path, name: str)
tests/tier3/test_interactions.py:23
↓ 17 callersFunction_tree
(root: Path, files: dict)
tests/tier3/test_plugin_prepare_review.py:27
↓ 17 callersMethod_verify_findings_with_llm
Run LLM second-pass verification on security findings. Findings the LLM classifies as false_positive with high confidence are downgra
src/skillevaluator/validators/security.py:6701
↓ 17 callersFunction_write_catalog
(path: Path, data: dict | None = None)
tests/embedding/test_catalog.py:71
↓ 17 callersFunction_write_skill
( tmp_path: Path, version_line: str = "", *, top_level_version: str = "", )
tests/validators/test_version.py:13
↓ 17 callersFunction_write_skill
( root: Path, frontmatter: str, body: str = "# Portable\n\n## Instructions\n\n1. Step\n\n## Examples\n\nEx
tests/validators/test_schema.py:1547
↓ 17 callersFunctionassess_publication
Return a shared, fail-closed publication assessment for all reporters.
src/skillevaluator/reporting/base.py:2387
↓ 17 callersFunctionbind_runtime_results
Bind a private run's output path without excluding authored source changes. The analyzed canonical identity remains intact. A separate staging gu
src/skillevaluator/tier3/plugin_prepare.py:278
↓ 17 callersFunctionbuild_clusters
Cluster chunks by pairwise cosine similarity using Union-Find.
src/skillevaluator/deduplication/intra_skill/semantic_clustering.py:90
↓ 17 callersMethodcatalog_self_entry
Return the catalog entry that records *target* itself, if any. Catalog paths are relative to a scan root the catalog does not record,
src/skillevaluator/embedding/registry.py:442
↓ 17 callersFunctioncompact_run_context
Trim the payload to the evidence the feedback chat may cite. Only the run id is included, not the run's host path. Absolute paths that end in
src/skillevaluator/evaluation/tier3_feedback_chat.py:955
↓ 17 callersFunctiondecoy_source_digest
Hash one stable runtime projection, rejecting unsafe or unstable trees. Secure staging traversal rejects links, hard-linked files, special nodes
src/skillevaluator/tier3/decoy_identity.py:41
↓ 17 callersFunctionfailure_reason
Return one short, deduplicated, redacted reason for a failing metric. ``include_results=False`` leaves out failing behavior results, for evaluato
src/skillevaluator/evaluation/tier3_findings.py:225
↓ 17 callersFunctionfind_skills_in_directory
Find live skill directories containing a supported manifest. Prunes configured scan exclusions and never validates linked directories. Extern
src/skillevaluator/utils/helpers.py:49
↓ 17 callersFunctionhttp_error
(code=429, body=b'{"error":{"code":"capacity_exceeded"}}', headers=None)
tests/tier3/test_judge_policies.py:89
↓ 17 callersFunctionload
(path)
tests/ci/test_publisher_workflow.py:21
↓ 17 callersMethodprint
(self, value)
tests/test_tier3_result_display.py:662
↓ 17 callersFunctionredact_secrets_in_log_line
Best-effort mask common key shapes in Layer 2 output text.
src/skillevaluator/tier3/eval_core/secret_redaction.py:45
↓ 17 callersFunctionrender_native_reports
Write all available views and return per-skill rendering diagnostics.
ci/native_reports.py:204
↓ 17 callersFunctionreport_text
Report text: whitespace collapsed, URL ``user:password@`` removed, credentials redacted, length bounded. URL userinfo goes after ``//`` and after
src/skillevaluator/validators/url_policy.py:664
↓ 17 callersFunctionresource_comparison
Compare complete matching batches; missing counters and baseline remain null.
src/skillevaluator/tier3/harbor/resource_usage.py:54
↓ 17 callersFunctionsummary
()
tests/ci/test_publish_reports.py:47
↓ 17 callersMethodto_legacy_string
Convert to legacy error string format for backward compatibility.
src/skillevaluator/models/result.py:241
↓ 16 callersFunction_agent
(*, baseline: bool = True, with_scores: dict[str, float | None] | None = None)
tests/reporting/test_dimension_explanations.py:25
↓ 16 callersMethod_argv
( self, tmp_path: Path, *, allow_net: bool = False, extra_ro: list[Pat
tests/test_harbor_local_sandbox.py:368
↓ 16 callersFunction_arm
Write one arm-level job; each outcome is ``(case, "scored" | "timeout")``.
tests/tier3/test_baseline_timeouts.py:169
↓ 16 callersFunction_changed
(path: Path, detail: str = "")
src/skillevaluator/tier3/harbor/secure_copy.py:305
↓ 16 callersFunction_checks
(result, name: str)
tests/validators/test_schema.py:1563
↓ 16 callersFunction_condition
(status: str, scored: int, expected: int = 2, errors: list[str] | None = None)
tests/reporting/test_tier3_incomplete_arms.py:51
↓ 16 callersFunction_marker_for
(*secrets: str)
tests/test_harbor_secure_docker_environment.py:60
↓ 16 callersFunction_one
Return the single match for *pattern*, failing loudly if the prose moved.
tests/test_metrics_reference_docs.py:88
↓ 16 callersFunction_ordinary_repository
(root: Path)
tests/test_artifact_applicability.py:48
↓ 16 callersFunction_pii
(result)
tests/validators/test_pii_waivers.py:75
↓ 16 callersFunction_prepared
(root: Path)
tests/tier3/test_plugin_execution.py:122
↓ 16 callersFunction_provider
(name: str, model: str)
tests/test_provider_agent_routing.py:26
↓ 16 callersFunction_request_chars
Characters of one review request, measured as the validator measures it.
tests/deduplication/intra_skill/test_intra_llm_analyzer.py:25
↓ 16 callersFunction_rooted
A clean skill in .agents/skills, which every harness but Claude Code scans.
tests/harness/test_overlay.py:43
↓ 16 callersFunction_runtime_skill_copy_ignore
Build a root-aware ignore callback for an agent-visible skill copy.
src/skillevaluator/tier3/harbor/adapter.py:861
↓ 16 callersFunction_scan
(tmp_path: Path, card: str, *, require: bool)
tests/test_benchmark_source_provenance.py:410
↓ 16 callersFunction_trajectory
(skill: str | None = None)
tests/reporting/test_activation_validation.py:18
↓ 16 callersFunction_two_agent_jobs
opencode's baseline times out on case-002; codex completes both arms. Returns launch errors.
tests/tier3/test_baseline_timeouts.py:193
↓ 16 callersFunction_view
(payload)
tests/reporting/test_telemetry_view.py:14
↓ 16 callersMethodbuild
(self)
src/skillevaluator/tier3/loose_parts.py:763
↓ 16 callersMethodcheck
(self, targets: Iterable[EndpointTarget])
src/skillevaluator/validators/endpoint_resolution.py:202
↓ 16 callersMethodexec_as_agent
( self, environment: BaseEnvironment, command: str, env: dict[str, str] | None
src/skillevaluator/tier3/harbor/local_agents.py:924
↓ 16 callersMethodfind_duplicates
Pairwise comparison of all indexed entries. Only pairs with cosine similarity >= threshold are returned, sorted by score descending.
src/skillevaluator/embedding/registry.py:372
← previousnext →401–500 of 24,741, ranked by callers