MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 2 callersMethod_validate_frontmatter
Parse .mdc file and validate YAML frontmatter against Pydantic schema.
src/skillevaluator/validators/rules_schema.py:148
↓ 2 callersFunction_validate_generated_json_structure
Reject trees that the bounded report loader cannot traverse.
src/skillevaluator/tier3/harbor/collector.py:1908
↓ 2 callersFunction_validate_graph
( value: object, *, max_nodes: int = MAX_STRUCTURED_NODES, max_collection_items: int = MAX_STR
src/skillevaluator/utils/structured_data.py:195
↓ 2 callersFunction_validate_hidden_fallback_tree
(stage: Path, manifests: tuple[_TreeManifest, ...])
src/skillevaluator/tier3/harbor/secure_copy.py:1931
↓ 2 callersFunction_validate_http_url
Allow explicit public provider endpoints, not local-file schemes.
src/skillevaluator/tier3/harbor/templates/eval.py:1139
↓ 2 callersFunction_validate_key_directory
(path: Path, metadata: os.stat_result)
src/skillevaluator/tier3/output_provenance.py:379
↓ 2 callersFunction_validate_native_host_env_mapping
Reject task-authored parent-env reads outside one staged role.
src/skillevaluator/tier3/harbor/adapter.py:5077
↓ 2 callersFunction_validate_points
(points: object, source: str, location: str)
src/skillevaluator/quality_config.py:153
↓ 2 callersFunction_validate_provider
(provider: str, *, variable: str)
src/skillevaluator/provider_config.py:888
↓ 2 callersFunction_validate_quality_messages
(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, *extra_args: str)
tests/test_commands.py:974
↓ 2 callersMethod_validate_single_rule
Run all schema validation checks on a single .mdc file.
src/skillevaluator/validators/rules_schema.py:96
↓ 2 callersMethod_validate_single_skill
Run the full 4-dimension quality analysis on one skill.
src/skillevaluator/validators/quality_score.py:402
↓ 2 callersMethod_validate_single_skill
(self, skill_path: Path)
src/skillevaluator/validators/rubric_eval.py:758
↓ 2 callersFunction_validate_text
(value: object, source: str, location: str)
src/skillevaluator/quality_config.py:165
↓ 2 callersFunction_validate_text_fields
(entry_id: str, name: object, description: object)
src/skillevaluator/embedding/registry.py:878
↓ 2 callersFunction_validate_timeout
(timeout_seconds: float)
src/skillevaluator/model_catalog.py:747
↓ 2 callersFunction_validate_tree_exact_stable
Sample opposite sibling orders to detect more incidental changes.
src/skillevaluator/tier3/harbor/secure_copy.py:1190
↓ 2 callersFunction_validate_windows_catalog_parent
(catalog_path: Path, *, create: bool)
src/skillevaluator/embedding/registry.py:1096
↓ 2 callersMethod_validate_with_applicability
Validate each discovered skill and total their SkillSpector applicability counters. The folder walker does not merge per-skill metadata, so t
src/skillevaluator/validators/security.py:3235
↓ 2 callersMethod_validate_workflow_manifest
Parse and validate workflow-rules.mdc frontmatter.
src/skillevaluator/validators/workflows_schema.py:174
↓ 2 callersFunction_validated_archive_member_path
(member_name: str, *, expected_root: str | None = None)
scripts/check_oss_boundary.py:923
↓ 2 callersFunction_validated_ci_context
Allow only bounded display text and HTTPS links in trusted CI chrome.
src/skillevaluator/reporting/html.py:2169
↓ 2 callersFunction_validated_pass_threshold
Return one finite unit-interval attempt threshold.
src/skillevaluator/tier3/harbor/runner.py:1863
↓ 2 callersFunction_validated_strict_read_roots
Filter shared trees and reject HOME-covering strict exceptions.
src/skillevaluator/tier3/harbor/local_sandbox.py:375
↓ 2 callersFunction_validated_vector_values
( vector: object, expected_dimension: int | None, *, context: str, allow_zero: bool, )
src/skillevaluator/embedding/client.py:422
↓ 2 callersMethod_value_reading
Return how the assignment rule reads the value after the name *match* (None: it masks none there). See _assignment_value_reading.
src/skillevaluator/tier3/eval_core/checks.py:4868
↓ 2 callersFunction_verdict_callout
(status: str)
src/skillevaluator/reporting/benchmark.py:1217
↓ 2 callersFunction_verdict_for_score
Map a 0-1 score to PASS / NEUTRAL / FAIL using the configured thresholds.
src/skillevaluator/evaluation/dimension_judge.py:290
↓ 2 callersFunction_verdict_policy
Expose the distinct task-attempt, dimension, and overall-lift gates.
src/skillevaluator/evaluation/tier3_report.py:5093
↓ 2 callersFunction_verifier_output_dir
Directory that receives skill_evaluator_reward.json (see write_reward_outputs).
src/skillevaluator/tier3/harbor/templates/eval.py:3076
↓ 2 callersFunction_verify_fallback_publication
(path: Path, expected: os.stat_result, *, kind: str)
src/skillevaluator/tier3/harbor/secure_copy.py:1950
↓ 2 callersFunction_view_rows
The rows as one harness and load mode loads them (see :data:`_VIEW_TYPES`). Claude Code applies one forced output style (the first in file order)
src/skillevaluator/plugin_context_cost.py:296
↓ 2 callersFunction_visible_html_lines
Return line-relative text that a raw HTML block would visibly render.
scripts/ci/check_public_benchmarks.py:1515
↓ 2 callersFunction_volume
(status: str)
tests/reporting/test_evidence_volume_rendering.py:35
↓ 2 callersFunction_wall_time_seconds
(value: float)
src/skillevaluator/tier3/harbor/trajectory_timing.py:206
↓ 2 callersFunction_windows_artifact_filesystem_reserve
Snapshot one filesystem's minimum free headroom for an operation.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:1096
↓ 2 callersFunction_windows_open_anchored_directory_chain
Pin an absolute directory from its volume anchor without following reparses.
src/skillevaluator/utils/secure_fs.py:832
↓ 2 callersMethod_with_capability_differences
``report`` with every piece that loads differently in the two arms counted as a mismatch. Equal bytes are not equal capability when Claude Co
src/skillevaluator/tier3/loose_parts.py:458
↓ 2 callersFunction_with_launch_defaults
``config`` with every ``${NAME:-default}`` in ``command`` and ``args`` replaced by its default. The same object comes back when nothing has a def
src/skillevaluator/validators/mcp_static.py:2611
↓ 2 callersMethod_with_missing_pieces
Expected pieces cannot vanish from every arm and disappear from the comparison.
src/skillevaluator/tier3/loose_parts.py:445
↓ 2 callersMethod_with_pii_waivers
Run ``scan`` of ``skill_path`` with the configured PII waivers, then report their outcomes once. The outcomes are added after the scan (and a
src/skillevaluator/validators/security.py:3214
↓ 2 callersFunction_with_requirements
Every ``--with`` (``-w``) requirement in ``options``; one value may list several, comma-separated.
src/skillevaluator/validators/mcp_static.py:2078
↓ 2 callersFunction_with_success_rate
``bucket`` with ``success_rate``, ``succeeded / (succeeded + failed)``: an unknown outcome counts as neither.
src/skillevaluator/tier3/eval_core/plugin_signals.py:4159
↓ 2 callersFunction_within_report_limits
Whether the payload fits the embedded size budget and the reporters' node, depth and text limits.
src/skillevaluator/evaluation/tier3_report.py:1810
↓ 2 callersFunction_without_allowances
( findings: Iterable[BoundaryFinding], allowances: Iterable[BoundaryAllowance], *, anchors_by_
scripts/check_oss_boundary.py:821
↓ 2 callersFunction_without_plain_patch_headers
Return *patch* without its file header lines, except those ``_keeps_patch_header`` keeps.
src/skillevaluator/tier3/eval_core/checks.py:1525
↓ 2 callersFunction_workflow_reference
(value: str)
ci/publish_reports.py:591
↓ 2 callersFunction_workspace_relative_path
Return the artifact path relative to the workspace root, or ``None``. The resolved path is tried first. A symlink inside the workspace whose
src/skillevaluator/reporting/sarif_reporter.py:154
↓ 2 callersFunction_workspace_skill_name
Reserve the effective skill name without requiring an embedding description.
src/skillevaluator/tier3/harbor/runner.py:3427
↓ 2 callersFunction_write
(path, value)
src/skillevaluator/tier3/grader_validation.py:418
↓ 2 callersFunction_write
(root: Path, relative: Path, payload: bytes)
ci/diagnostics.py:150
↓ 2 callersFunction_write
(path: Path, content: str)
ci/report_views.py:298
↓ 2 callersFunction_write_agent_configs
Use Harbor's agent integrations and provider-native environment variables.
src/skillevaluator/tier3/harbor/adapter.py:3954
↓ 2 callersFunction_write_all
(descriptor: int, payload: bytes)
src/skillevaluator/reporting/base.py:172
↓ 2 callersFunction_write_all
(descriptor: int, payload: bytes)
src/skillevaluator/embedding/registry.py:1035
↓ 2 callersMethod_write_artifact
An advisory artifact cannot prevent the others from being collected.
src/skillevaluator/tier3/plugin_comparisons.py:337
↓ 2 callersFunction_write_artifact_manifest
(trial_out: Path, manifest: dict[str, Any])
src/skillevaluator/tier3/harbor/collector.py:1287
↓ 2 callersFunction_write_atomic
(target: Path, data: bytes, mode: int)
src/skillevaluator/utils/scanner_install.py:76
↓ 2 callersFunction_write_catalog_atomically
(catalog_path: Path, payload: bytes)
src/skillevaluator/embedding/registry.py:1222
↓ 2 callersFunction_write_catalog_summary
Write catalog-context verdicts and counts; linked leaf reports retain single-skill scope.
src/skillevaluator/cli.py:1657
↓ 2 callersFunction_write_default_environment_dockerfile
( env_dir: Path, *, base_image: str, agent_config_lines: list[str], include_input: bool,
src/skillevaluator/tier3/harbor/adapter.py:4509
↓ 2 callersFunction_write_dockerfile
Generate a Dockerfile that installs skills into the container. Environment resolution order: 1. ``skill_path/evals/environment/Dockerfile``
src/skillevaluator/tier3/harbor/adapter.py:4189
↓ 2 callersFunction_write_evals_config
(evals_dir: Path, *, grading_mode: str, task_source: str)
src/skillevaluator/tier3/commands.py:189
↓ 2 callersFunction_write_files
(root: Path, files: dict[str, str])
tests/validators/test_artifact_dir_exclusion.py:398
↓ 2 callersFunction_write_grader_file
(path: Path, *, language: str)
src/skillevaluator/tier3/commands.py:316
↓ 2 callersFunction_write_job
( job: Path, accuracies: list[float], judges: list[tuple[str | None, ...]], *, native: boo
tests/test_judge_lift_integrity.py:37
↓ 2 callersFunction_write_job
(jobs: Path, variant: str, rewards: list[dict])
tests/test_harbor_skill_activation.py:108
↓ 2 callersFunction_write_job
(job_dir: Path, scores: dict[str, float])
tests/tier3/test_skip_with_skill.py:46
↓ 2 callersFunction_write_job_result
(job_dir: Path, trial_names: list[str])
tests/tier3/test_skip_with_skill.py:54
↓ 2 callersFunction_write_json
(path: Path, payload: dict)
ci/run_tier1.py:205
↓ 2 callersFunction_write_judge_attempts
(attempts)
src/skillevaluator/tier3/harbor/templates/eval.py:3123
↓ 2 callersFunction_write_lowercase_manifest_skill
(parent: Path, name: str)
tests/test_precommit.py:514
↓ 2 callersFunction_write_mcp_skill_fixture
Create a minimal skill directory with SKILL.md, evals.json, optional config.yml, and mcp_servers.toml.
tests/test_harbor_gke_mode.py:1235
↓ 2 callersFunction_write_native_skill
(tmp_path: Path)
tests/test_harbor_adapter_safety.py:101
↓ 2 callersFunction_write_nested_runtime_skill
(wrapper: Path, package: str)
tests/test_harbor_runtime_skill_isolation.py:74
↓ 2 callersFunction_write_repo_context_fixture
(repo: Path)
tests/test_harbor_adapter_safety.py:113
↓ 2 callersFunction_write_report_atomically
(output_path: Path, payload: bytes)
src/skillevaluator/reporting/base.py:404
↓ 2 callersFunction_write_scored_job
(job_dir: Path, reward: dict[str, object])
tests/test_harbor_unscored_grading_policy.py:60
↓ 2 callersFunction_write_scored_run
Write one complete opencode agent's arm summaries into a run directory.
tests/reporting/test_tier3_completed_agent_docs.py:99
↓ 2 callersFunction_write_skill
(tmp_path: Path)
tests/test_harbor_claude_background_tasks.py:32
↓ 2 callersMethod_write_skill
(root: Path, name: str, *, author_line: str, id_line: str | None)
tests/validators/test_schema.py:1418
↓ 2 callersFunction_write_stdin_trajectory
(content: str)
tests/tier3/test_trajectory_timing.py:297
↓ 2 callersFunction_write_successful_job_result
(job_dir: Path, trial_name: str)
tests/test_harbor_runner_status.py:1417
↓ 2 callersFunction_yaml_flow_tokens
Yield ``(kind, token, depth, end)`` for the flow collection that opens at ``text[start]``. A flow collection may continue over lines whose keys a
src/skillevaluator/tier3/harbor/templates/eval.py:4225
↓ 2 callersFunction_yaml_line_end
(text: str, position: int)
src/skillevaluator/tier3/harbor/templates/eval.py:4214
↓ 2 callersFunction_yaml_unquote
Return the value of the single- or double-quoted scalar ``raw`` (quotes included).
src/skillevaluator/tier3/harbor/templates/eval.py:4219
↓ 2 callersFunction_zero_token_notice
(source: str)
src/skillevaluator/reporting/dashboard_data.py:299
↓ 2 callersMethodaclose
(self)
src/skillevaluator/tier3/mcp_proof.py:497
↓ 2 callersFunctionactive_runtime
The private runtime hooks of the current run, or ``None`` for an ordinary run.
src/skillevaluator/tier3/plugin_execution.py:57
↓ 2 callersFunctionadapter_for
(agent: str)
src/skillevaluator/tier3/plugin_native.py:2159
↓ 2 callersFunctionadd
(text: str, first: int, last: int)
src/skillevaluator/deduplication/utils/chunker.py:88
↓ 2 callersMethodadd_message
(self, message: str)
tests/validators/test_plugin_dependencies.py:330
↓ 2 callersMethodadmit_directory
Apply the directory rules; return whether the walker should descend.
src/skillevaluator/utils/secure_fs.py:1012
↓ 2 callersMethodadmit_file
Apply the rules for a non-directory entry and keep it when selected. Redirects fail closed except the exact ``CLAUDE.md -> AGENTS.md`` alias,
src/skillevaluator/utils/secure_fs.py:1037
↓ 2 callersFunctionagent_eval_baseline_not_applicable
Return whether the baseline recorded a dimension as not applicable instead of scoring it. Only the skill metrics qualify. Any other metric is not
src/skillevaluator/reporting/base.py:1153
↓ 2 callersFunctionagent_eval_report_serialization_limits
Return the shared depth and node limits for embedded Tier 3 payloads.
src/skillevaluator/reporting/base.py:498
↓ 2 callersFunctionagent_eval_report_text_limit
Return the aggregate UTF-8 text budget for embedded Tier 3 payloads.
src/skillevaluator/reporting/base.py:503
↓ 2 callersFunctionagent_label
(name: str | None)
src/skillevaluator/reporting/benchmark.py:1551
↓ 2 callersFunctionaggregate_execution_error_details
Sum condition occurrence counts while deduplicating only displayed text.
src/skillevaluator/tier3/harbor/report_data.py:631
↓ 2 callersFunctionaggregate_job
()
ci/feedback.py:978
← previousnext →4,801–4,900 of 24,741, ranked by callers