MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 11 callersFunction_png_bytes
Return the 1x1 PNG that the referenced-image SkillSpector fixtures scanned.
tests/validators/test_security.py:8865
↓ 11 callersMethod_print_agent_eval_tables
Print Tier 3 agent evaluation results.
src/skillevaluator/reporting/cli.py:454
↓ 11 callersMethod_process_semgrep_output
Parse Semgrep JSON output and report findings. *skill_path* is the resolved skill root; it lets parse-error paths be shown relative to it.
src/skillevaluator/validators/code_risk.py:363
↓ 11 callersFunction_render
(renderable: Any, *, width: int = 80)
tests/test_tier3_progress.py:285
↓ 11 callersFunction_report
(root)
tests/reporting/test_applicability_saved_runs.py:83
↓ 11 callersFunction_report_roots
Return the scan and repository roots SARIF and JUnit paths are made relative to. The repository root is the Git root containing *target_path*, so
src/skillevaluator/cli.py:624
↓ 11 callersFunction_reward
(entry_id, findings)
tests/tier3/test_harbor_security_attribution.py:10
↓ 11 callersFunction_run_gym
Run ``gym eval run`` in its own process group; return its exit code, or None on timeout. Gym's output reaches ``log`` through the secret redactor
src/skillevaluator/tier3/nemo_gym/runner.py:578
↓ 11 callersFunction_run_result
( check_id: str, name: str, description: str, validator: ValidatorRunner,
src/skillevaluator/tier1/commands.py:375
↓ 11 callersMethod_run_semgrep
Run Semgrep security scanner.
src/skillevaluator/validators/code_risk.py:312
↓ 11 callersFunction_safe_text
(value: Any, limit: int = _MAX_LABEL_CHARS)
src/skillevaluator/tier3/eval_core/plugin_signals.py:1731
↓ 11 callersFunction_security_details
(*findings: dict[str, Any], reason: str = DESTRUCTIVE_REASON)
tests/reporting/test_tier3_failure_reasons.py:65
↓ 11 callersFunction_short_description_issues
(result)
tests/validators/test_quality_score.py:1795
↓ 11 callersFunction_shown
A bounded, quoted rendering of a manifest value for messages.
src/skillevaluator/plugin_format_schema.py:361
↓ 11 callersFunction_skill
(name: str)
tests/test_openclaw_plugin_discovery.py:28
↓ 11 callersFunction_skill
(root: Path, name: str, *, skill_name: str | None = None)
tests/tier3/test_decoys.py:29
↓ 11 callersFunction_skill_with_config
(tmp_path, text)
tests/tier3/test_evals_config.py:24
↓ 11 callersFunction_source
(tmp_path: Path, server: dict, *, manifest_dir=".claude-plugin", local_file=False)
tests/tier3/test_native_mcp_policy.py:19
↓ 11 callersFunction_source_result
(tmp_path: Path, checks: list[CheckResult], *, blocking: bool = True)
tests/reporting/test_tier3_source_validation_reporting.py:70
↓ 11 callersFunction_successful_summary_payload
(agent: str = "opencode")
tests/test_results_location_legacy_compatibility.py:84
↓ 11 callersFunction_valid_data
(**overrides)
tests/models/test_plugin.py:12
↓ 11 callersFunction_write_manifest
(dir_path: Path, body: str, name: str = "agent_plugin.yaml")
tests/validators/test_plugin_schema.py:28
↓ 11 callersFunction_write_redacted_text_copy
Copy a Harbor text artifact while masking common credential shapes.
src/skillevaluator/tier3/harbor/collector.py:1297
↓ 11 callersMethod_write_skill
Write a minimal SKILL.md with the given author (or no metadata block).
tests/validators/test_schema.py:951
↓ 11 callersFunction_write_summary
(agent_dir: Path)
tests/reporting/test_report_data_bounds.py:20
↓ 11 callersFunction_write_trial
(agent_dir: Path, trial_name: str, reward: dict, trajectory: dict | None = None)
tests/reporting/test_report_data_bounds.py:38
↓ 11 callersFunctionbudget_incomplete_reason
(metadata: object)
src/skillevaluator/tier3/harbor/run_budget.py:487
↓ 11 callersFunctioncheck_workflow_order
Check whether the agent read SKILL.md before executing scripts. Also treats Claude Code ``Skill`` tool activation as a valid "read" step.
src/skillevaluator/tier3/eval_core/checks.py:7215
↓ 11 callersFunctionchunk_markdown
Split markdown into heading-level sections with line tracking.
src/skillevaluator/deduplication/utils/chunker.py:121
↓ 11 callersFunctionchunk_shell
Extract function bodies and comment blocks from shell scripts.
src/skillevaluator/deduplication/utils/chunker.py:306
↓ 11 callersFunctionclassify_endpoint_host
Classify an MCP URL host as a metadata or private endpoint, network-free. Only IP literals (including IPv4-mapped/embedded IPv6 and decimal, hex,
src/skillevaluator/validators/mcp_static.py:2932
↓ 11 callersFunctioncollected
(text, extension=".md")
tests/deduplication/utils/test_chunk_bounds.py:14
↓ 11 callersFunctioncompare_configurations
Require complete per-case, per-agent evidence before calculating a score. Every case has equal weight; attempts are averaged within a case first.
src/skillevaluator/tier3/interactions.py:133
↓ 11 callersMethodcontext_cost
(self, **kwargs: Any)
src/skillevaluator/plugin_inventory.py:187
↓ 11 callersFunctionemit
(result)
tests/reporting/test_retained_tiers.py:15
↓ 11 callersFunctionfacts_for
Facts per skill, with the catalog they are installed in. A skill installed in a skills folder (e.g. ``.claude/skills``, or an OpenClaw workspace
src/skillevaluator/harness/overlay.py:665
↓ 11 callersMethodfinish
(self)
src/skillevaluator/tier3/harbor/stream_redaction.py:293
↓ 11 callersMethodgetPixelForValue
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 11 callersMethodinfo
The ``loose_parts`` JSON block: what the arm staged, the content hashes, glue, and load evidence. ``content`` defaults to :meth:`content_repo
src/skillevaluator/tier3/loose_parts.py:493
↓ 11 callersFunctionio_api
()
tests/reporting/test_private_runtime_replay.py:14
↓ 11 callersFunctionis_advisory_agent_eval_skip
Return whether a Tier 3 result records a non-blocking skipped run.
src/skillevaluator/reporting/base.py:412
↓ 11 callersMethodis_error
Determine if this severity should cause validation failure. Args: fail_on_medium: Treat MEDIUM severity as error fail
src/skillevaluator/models/result.py:43
↓ 11 callersFunctionis_retriable_exception
Return True if an exception represents a transient failure eligible for retry.
src/skillevaluator/inference/retry.py:189
↓ 11 callersFunctionmake_gke_env
Provide a factory for SkillEvaluatorGKEEnvironment wired to fake CoreV1Api and NetworkingV1Api.
tests/test_harbor_gke_environment.py:84
↓ 11 callersFunctionnormalize_dataset_entries
Normalize supported eval dataset shapes to runtime fields.
src/skillevaluator/tier3/dataset_utils.py:115
↓ 11 callersFunctionnormalize_optional_reference
(value: Any)
src/skillevaluator/tier3/eval_core/optional_reference.py:11
↓ 11 callersMethodomit
(self, section: str, count: int = 1)
src/skillevaluator/evaluation/tier3_report.py:183
↓ 11 callersMethodrender_all
Render all results to string with summary table.
src/skillevaluator/reporting/cli.py:170
↓ 11 callersFunctionsanitize_skillspector_llm_provenance
Return the canonical safe SkillSpector provenance record, or ``None`` if invalid.
src/skillevaluator/skillspector_provenance.py:255
↓ 11 callersFunctionselect_decoys
Rank local skills from a description-mode Tier 2 catalog. Exact target copies, duplicate identities, and negative-similarity matches are inel
src/skillevaluator/tier3/decoys.py:168
↓ 11 callersMethodservice_download_file
( self, source_path: str, target_path: Path | str, *, service: str | N
src/skillevaluator/tier3/harbor/secure_docker_environment.py:2394
↓ 11 callersFunctionskill_entry
(path, *, fingerprint=None, content_type="skill")
tests/deduplication/plugin/test_private_dedup.py:116
↓ 11 callersMethodstart
(self, plan: Tier3RunPlan)
src/skillevaluator/tier3/harbor/progress.py:236
↓ 11 callersFunctiontier2_scope_views
Keep missing completion/count metadata unknown; never rewrite results. Workflow status describes scope, while a failed gate may still have finish
src/skillevaluator/reporting/tier2_views.py:57
↓ 11 callersFunctiontimed
(entry="case", llm=10.0, tool=2.0, source="step_timestamps", prompt=100, completion=20)
tests/tier3/test_resource_usage.py:82
↓ 11 callersFunctiontoday_utc
Return today's date in UTC, the calendar waiver review windows use.
src/skillevaluator/validators/pii_waivers.py:96
↓ 11 callersMethodtrace
(self, trial_root: Path)
src/skillevaluator/tier3/plugin_trace_artifacts.py:508
↓ 11 callersFunctionvalidate_environment_kwargs
Validate non-secret Harbor constructor kwargs for safe argv forwarding.
src/skillevaluator/tier3/evals_config.py:700
↓ 11 callersFunctionvalidate_skillevaluators
Validate a skill's evals/ directory against the spec. Returns a list of check results — one per spec entry, plus any additional checks (datas
src/skillevaluator/tier3/evals_spec.py:351
↓ 11 callersFunctionwrite_job_from_rollouts
Write one collector job directory from Gym rollouts and return the run errors. ``task_case_ids`` maps each staged task folder name to its evals.j
src/skillevaluator/tier3/nemo_gym/results.py:140
↓ 10 callersFunctionZ
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 10 callersFunction_agent
(tmp_path)
tests/test_hermes_agent.py:66
↓ 10 callersFunction_agent_quality_verdict
Classify one supported agent by required checks, then dimensions. A dimension below the neutral threshold fails the agent even when another d
src/skillevaluator/evaluation/tier3_report.py:4640
↓ 10 callersFunction_anthropic_response
(text: str, *, stop_reason: str = "end_turn")
tests/inference/test_llm_client.py:1006
↓ 10 callersFunction_arm
Write one arm: ``trials`` are (trial directory, (prompt, completion, cached), has a reward).
tests/reporting/test_tier3_token_usage_reports.py:86
↓ 10 callersFunction_assert_no_secret
(value: str | bytes)
tests/ci/test_secret_redaction.py:44
↓ 10 callersMethod_build_registry_with_entries
( self, client: EmbeddingClient, entries: dict[str, list[float]], **limits: int )
tests/embedding/test_registry.py:389
↓ 10 callersFunction_capture_launch
Invoke the dashboard command with Streamlit present and the child process faked.
tests/reporting/test_dashboard_cli.py:56
↓ 10 callersFunction_command_text
(tool_call: dict[str, Any])
src/skillevaluator/tier3/eval_core/checks.py:896
↓ 10 callersFunction_custom_backend
Load an installed ``--env-mode`` backend and check the host variables it forwards.
src/skillevaluator/tier3/harbor/runner.py:1370
↓ 10 callersFunction_detect_from_file
Detect content type from a file path.
src/skillevaluator/cli_core.py:37
↓ 10 callersMethod_each
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 10 callersFunction_fake_engine
(monkeypatch, scores=(0.50, 0.75, 0.70, 0.65), *, on_run=None)
tests/tier3/test_interactions.py:356
↓ 10 callersFunction_findings_kpi
Return the Findings KPI value and its reconciliation detail line ("" when absent).
tests/reporting/test_reporters.py:805
↓ 10 callersFunction_helps
(ids: tuple[str, ...] = _IDS, *, negative: bool = True)
tests/test_tier3_next_steps.py:33
↓ 10 callersFunction_hook_rows
(result: ValidationResult)
tests/validators/test_plugin_static_risk_review_fixes.py:69
↓ 10 callersFunction_incomplete_result
()
tests/reporting/test_incomplete_scans.py:39
↓ 10 callersFunction_integer
(value, minimum=0)
ci/publish_reports.py:115
↓ 10 callersMethod_is_generated_artifact_issue
Return True when a skillspector issue points at generated output.
src/skillevaluator/validators/security.py:5371
↓ 10 callersFunction_limit
(message: str)
src/skillevaluator/utils/structured_data.py:95
↓ 10 callersFunction_mask_as
(copy: str, text: str)
tests/tier3/test_security_finding_evidence.py:3599
↓ 10 callersMethod_merge_env
Merge environment variables while ensuring _exec_env_overlays exists for object.__new__ instances.
src/skillevaluator/tier3/harbor/gke_environment.py:1188
↓ 10 callersFunction_package
(root: Path, openclaw: Any, files: dict[str, Any] | None = None, **fields: Any)
tests/test_openclaw_plugin_discovery.py:322
↓ 10 callersFunction_parse_in_order
Parse a skill, then list its scripts in `order`, because the directory listing order is not defined.
tests/tier3/test_dataset_primary_script.py:37
↓ 10 callersFunction_paths_equivalent
Return whether two paths identify the same cross-platform path.
src/skillevaluator/tier3/harbor/adapter.py:547
↓ 10 callersFunction_publication_ready_results
Return a result set whose unmodified evidence is publication-complete.
tests/reporting/test_advisory_skip_verdict.py:239
↓ 10 callersFunction_recording_console
(width: int = 200)
tests/reporting/test_rich_markup_escaping.py:75
↓ 10 callersFunction_reliability_issues
(result)
tests/validators/test_quality_score.py:1534
↓ 10 callersFunction_render_paired_comparison
(paired_comparison: object)
tests/reporting/test_unified_tier3_report.py:3315
↓ 10 callersFunction_replay
Validate and process one report as a static stage does.
tests/validators/test_skillspector_hook_surface.py:294
↓ 10 callersFunction_resolve_file_report_output_location
Resolve and reserve report storage only when a file reporter needs it.
src/skillevaluator/cli.py:637
↓ 10 callersMethod_resolved_config
Resolve and cache the selected public provider configuration.
src/skillevaluator/inference/client.py:573
↓ 10 callersFunction_reward
(score: float = 0.0, *, passed: bool | None = False, **activation: object)
tests/test_harbor_skill_activation.py:19
↓ 10 callersFunction_rule_ids
(skill: Path, harness: str, config: dict | None = None)
tests/harness/test_overlay.py:924
↓ 10 callersFunction_run
( monkeypatch: pytest.MonkeyPatch, jobs_dir: Path, job_name: str = "demo-opencode-with", expec
tests/test_harbor_runner_status.py:2310
↓ 10 callersFunction_run
Claude Code's baseline times out on case-2; Codex completes both arms.
tests/reporting/test_tier3_token_usage_reports.py:137
↓ 10 callersFunction_scripted_hub
Return a call_public_llm stub replaying *responses* and recording calls.
tests/tier3/test_judge_parse_robustness.py:307
↓ 10 callersFunction_security_without_llm
()
tests/reporting/test_tier3_validate_cases.py:302
↓ 10 callersFunction_short
(text: object, limit: int = _MAX_REASON_CHARS)
src/skillevaluator/tier3/case_summary.py:958
↓ 10 callersFunction_skill
(path: Path, *, skill_name: str | None = None)
tests/tier3/test_decoy_workspace.py:23
← previousnext →801–900 of 24,741, ranked by callers