MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 1 callersMethod_validate_single_skill
Run Gitleaks scan on a single skill directory or file.
src/skillevaluator/validators/secrets.py:95
↓ 1 callersMethod_validate_single_workflow
Run all validation checks on a single workflow directory.
src/skillevaluator/validators/workflows_schema.py:109
↓ 1 callersMethod_validate_skillspector_issue
Validate every nested issue field consumed by the report converter.
src/skillevaluator/validators/security.py:4975
↓ 1 callersFunction_validate_source
(source: Path, metric: str)
src/skillevaluator/tier3/interactions.py:263
↓ 1 callersFunction_validate_tier3_reports
Run ``validate --tier3`` after a passing Tier 1; return its exit code, JSON ``tier3`` object and HTML report. ``tier3`` stands in for the live ru
tests/reporting/test_tier3_completed_agent_docs.py:417
↓ 1 callersFunction_validate_tree_exact_pass
Verify one ordered, post-order-stable exact-tree pass.
src/skillevaluator/tier3/harbor/secure_copy.py:1098
↓ 1 callersFunction_validate_windows_entry_snapshot
( metadata: os.stat_result, handle_metadata: _WindowsHandleMetadata, relative_path: Path, )
src/skillevaluator/utils/secure_fs.py:1506
↓ 1 callersFunction_validate_windows_path_component
Reject Win32 normalization aliases, device names, ADS, and invalid UTF-16.
src/skillevaluator/utils/secure_fs.py:624
↓ 1 callersFunction_validate_windows_read_file_handle
Require one selected Windows handle to be regular, single-link, and no-follow.
src/skillevaluator/utils/secure_fs.py:814
↓ 1 callersFunction_validate_workflow_provenance
(api, repository: str, run: dict)
ci/publish_reports.py:618
↓ 1 callersFunction_validated_case_id_by_task_selector
Copy one trusted staged-selector mapping after rejecting ambiguity.
src/skillevaluator/tier3/harbor/collector.py:418
↓ 1 callersFunction_validated_windows_tar_member
Return one collision-safe Windows pathname or reject the member.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:1006
↓ 1 callersFunction_value_contains_protected_value
Match protected values inside structurally retained Compose values.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:280
↓ 1 callersFunction_value_produced
(value: str, call: _Call)
src/skillevaluator/tier3/eval_core/plugin_signals.py:4497
↓ 1 callersFunction_value_received
The value is in the consumer's arguments, compared as parsed values (quotes and backslashes are data).
src/skillevaluator/tier3/eval_core/plugin_signals.py:4501
↓ 1 callersFunction_values
(name: str)
src/skillevaluator/tier3/eval_core/plugin_signals.py:4851
↓ 1 callersMethod_verbs_at
(self, i: int)
src/skillevaluator/evaluation/dataset_case_removal.py:270
↓ 1 callersFunction_verdict
Read the quality chip and its reason inside the hero, without trailing page content.
tests/reporting/test_tier3_token_usage_reports.py:605
↓ 1 callersFunction_verdict_evidence_lines
Render the rule and the evidence that produced *status*. Every statement here is read from ``assessment`` — the same object ``_overall_status
src/skillevaluator/reporting/benchmark.py:1292
↓ 1 callersFunction_verdict_from_lift
(lift: float | None)
src/skillevaluator/evaluation/tier3_report.py:4629
↓ 1 callersFunction_verified_comment
(api, repository: str, number: int, identifier: int, bot: str, body: str)
ci/comment_report.py:151
↓ 1 callersFunction_verified_windows_taskkill_path
Resolve a regular, non-reparse taskkill executable inside System32.
src/skillevaluator/tier3/harbor/runner.py:3761
↓ 1 callersFunction_verifier_git_provenance
Disclose source identity only when every actual verifier file is tracked.
src/skillevaluator/tier3/grader_validation.py:464
↓ 1 callersMethod_verify_in_pod_metadata_isolation
Verify from the trusted probe container that GCE/GKE metadata endpoints are unreachable.
src/skillevaluator/tier3/harbor/gke_environment.py:1090
↓ 1 callersFunction_verify_pr
(api, repository: str, number: int, sha: str)
ci/comment_report.py:92
↓ 1 callersFunction_verify_published_node
(parent_descriptor: int, name: str, node: _StagedNode)
src/skillevaluator/tier3/harbor/secure_copy.py:804
↓ 1 callersFunction_verify_run
Reject superseded attempts, including reruns started while rendering feedback.
ci/comment_report.py:68
↓ 1 callersFunction_verify_windows_open_path
(descriptor: int, catalog_path: Path)
src/skillevaluator/embedding/registry.py:1173
↓ 1 callersFunction_version_manager_env
Where the caller's version managers live, so a shim on PATH still finds the real claude.
src/skillevaluator/validators/claude_plugin_validate.py:147
↓ 1 callersFunction_version_tuple
Parse a prevalidated version without allowing conversion failures to escape.
src/skillevaluator/validators/version.py:45
↓ 1 callersFunction_versioned_name
The interpreter under a versioned name, as distributions install it. Debian ships ``perl5.38.2`` beside ``perl`` and every platform ships ``p
scripts/script_invocation_differential.py:89
↓ 1 callersFunction_view_summary
(rows: list[CostRow], harness: str, load_mode: str)
src/skillevaluator/plugin_context_cost.py:350
↓ 1 callersFunction_visible_forms
Add the forms another check can echo for a value with invisible characters.
src/skillevaluator/utils/secret_redaction.py:62
↓ 1 callersFunction_visit
(node: Any, depth: int)
src/skillevaluator/tier3/harbor/adapter.py:3282
↓ 1 callersFunction_wait_for_harbor_leader
Wait up to ``timeout_seconds`` for Harbor to exit, or until ``end_grace`` is set.
src/skillevaluator/tier3/harbor/runner.py:3797
↓ 1 callersMethod_wait_for_pod_ready
Wait for pod readiness while failing fast if the metadata isolation init container fails.
src/skillevaluator/tier3/harbor/gke_environment.py:969
↓ 1 callersFunction_wait_for_windows_taskkill
Wait up to ``timeout_seconds`` for ``taskkill`` to exit, or until ``end_grace`` is set. The wait is cut into short slices so a set ``end_grace``
src/skillevaluator/tier3/harbor/runner.py:3815
↓ 1 callersFunction_walk_for_invocation
Whether a shell command invokes ``expected_script``. Credit is given only for a recognised way of running a script: the script invoked direct
src/skillevaluator/tier3/eval_core/script_invocation.py:2783
↓ 1 callersFunction_walk_posix
Admit every entry below ``root`` through held directory descriptors, never following links.
src/skillevaluator/utils/secure_fs.py:1171
↓ 1 callersFunction_walk_pruned_folders
Names-only walk that finds the folders the whole-tree walk prunes and the links inside them. The whole-tree verification (``verify_plugin_tree``)
src/skillevaluator/plugin_paths.py:565
↓ 1 callersFunction_walk_tokens
Yield tokens and their descendants in document order.
src/skillevaluator/validators/markdown.py:217
↓ 1 callersFunction_walk_windows
Admit every entry below ``root`` through pinned native handles, never following reparse points.
src/skillevaluator/utils/secure_fs.py:1308
↓ 1 callersFunction_wants_color
Decide whether to emit ANSI color, mirroring Click/rich conventions.
src/skillevaluator/cli_help.py:126
↓ 1 callersFunction_warn_unknown_keys
(data: dict[str, Any], known: set[str], context: str, source: str)
src/skillevaluator/validators/policy.py:205
↓ 1 callersFunction_weighted
( scores: dict[str, Any], evaluators: list[str], weights: list[float], *, not_applicable:
src/skillevaluator/evaluation/tier3_report.py:4562
↓ 1 callersFunction_weighted_mean
( value_of: Callable[[str], object], evaluators: Iterable[str], weights: Iterable[object], )
src/skillevaluator/tier3/harbor/metrics.py:700
↓ 1 callersFunction_weighted_signals
(config: dict[str, Any])
src/skillevaluator/reporting/benchmark.py:2125
↓ 1 callersFunction_welcome_banner
Brand interactive root help without adding output to command execution.
src/skillevaluator/cli_help.py:80
↓ 1 callersFunction_what_to_fix
( result: Mapping[str, Any], *, run_dir: Path, skill_path: Path | None, use_llm: bool )
src/skillevaluator/tier3/case_summary.py:1439
↓ 1 callersFunction_whatwg_domain_ok
Whether the WHATWG host parser accepts ``raw`` as a special-scheme domain or IPv4 host.
src/skillevaluator/plugin_format_schema.py:522
↓ 1 callersFunction_whatwg_hostname
The host a WHATWG client reads from ``url``; ``None`` when it finds none or cannot parse the URL.
src/skillevaluator/validators/mcp_static.py:435
↓ 1 callersFunction_whatwg_ipv4_number
One dotted part of a WHATWG IPv4 host (decimal, ``0x`` hex, or leading-zero octal), else ``None``.
src/skillevaluator/plugin_format_schema.py:487
↓ 1 callersFunction_whatwg_ipv4_ok
``None`` when ``host`` does not end in a number (a domain); else whether it is a valid WHATWG IPv4 host.
src/skillevaluator/plugin_format_schema.py:502
↓ 1 callersFunction_whatwg_ipv6_ok
(text: str)
src/skillevaluator/plugin_format_schema.py:550
↓ 1 callersMethod_whole_root
Copy and compare the full arm's resource tree without activating a plugin loader.
src/skillevaluator/tier3/loose_parts.py:1190
↓ 1 callersMethod_willRender
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunction_wilson_z_in_source
Read the ``z`` literal out of ``_wilson_score_interval`` itself.
tests/test_metrics_reference_docs.py:242
↓ 1 callersFunction_window_opener
The declared skill or command a call switched the agent to, or ``None``. Only a call that activated exactly one declared plugin skill or command,
src/skillevaluator/tier3/eval_core/plugin_signals.py:3293
↓ 1 callersFunction_windows_artifact_inode_budget
Return a headroom-preserving inode budget when the host exposes one.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:1106
↓ 1 callersFunction_windows_enumerate_pinned_directory_names
Enumerate names by path only while native handles pin every path component.
src/skillevaluator/utils/secure_fs.py:1530
↓ 1 callersFunction_windows_final_path
(descriptor: int)
src/skillevaluator/embedding/registry.py:1150
↓ 1 callersFunction_windows_final_path
Return the kernel-resolved path for an open Windows file descriptor.
src/skillevaluator/deduplication/utils/skill_collector.py:278
↓ 1 callersFunction_windows_open_discovery_handle
Open one authored entry without following it, including exact alias reparses.
src/skillevaluator/utils/secure_fs.py:1437
↓ 1 callersFunction_windows_open_handle
( path: Path, *, access: int, share: int, disposition: int, flags: int, )
src/skillevaluator/utils/secure_fs.py:594
↓ 1 callersFunction_windows_system_directory
Return the Windows system directory without consulting ambient environment.
src/skillevaluator/tier3/harbor/runner.py:3740
↓ 1 callersFunction_with_authority_slashes
A special-scheme ``url`` trimmed as WHATWG trims it, with the run of ``/`` and ``\\`` after the scheme as ``//``. A WHATWG client reads ``https:/
src/skillevaluator/validators/url_policy.py:858
↓ 1 callersFunction_with_host_detail
``<agent detail>; host probe: <pre-run detail>`` (the pre-run part kept once on a re-run).
src/skillevaluator/tier3/mcp_proof.py:892
↓ 1 callersFunction_with_incomplete_baseline
Return ``agent`` for the score table, with a failed baseline arm in place of its baseline scores. A baseline arm that ran but failed has no basel
src/skillevaluator/reporting/benchmark.py:1519
↓ 1 callersFunction_with_plugin_mcp_servers
Per-declared-server call counts summed over every agent's with-plugin arm. Each ``by_server`` key maps to at most one declared server through the
src/skillevaluator/tier3/mcp_proof.py:796
↓ 1 callersFunction_with_plugin_names
``declared`` with the plugin name, taken from the wrapper package name when not given.
src/skillevaluator/tier3/eval_core/plugin_signals.py:4747
↓ 1 callersFunction_with_security_defaults
The shipped command/argument/env defaults the client actually applies, without reading host env. Codex passes these strings literally. Other supp
src/skillevaluator/validators/mcp_static.py:2631
↓ 1 callersFunction_with_skill_tolerance_rejection
Check that the with-skill arm is complete and clean (T1); None when it is. ``runtime_failures`` and ``trial_failures`` are the active (unrecovere
src/skillevaluator/tier3/harbor/collector.py:7404
↓ 1 callersFunction_with_trial_identity
Return ``rewards`` with the trial directory as the identity of every row that has no usable one. A failed arm keeps each scored reward twice: as
src/skillevaluator/tier3/harbor/report_data.py:819
↓ 1 callersFunction_withheld_arm_blockers
Name each arm whose scores the report withholds: its usage is real but not comparable. A failed arm reuses the coverage notice's short form, such
src/skillevaluator/tier3/token_usage.py:477
↓ 1 callersFunction_withheld_attempt_usage
Return the token usage of scored attempts whose arm's scores are not reported. One row per logical attempt. As in report trials, an attempt score
src/skillevaluator/tier3/harbor/report_data.py:844
↓ 1 callersFunction_within_one_edit
One insertion, deletion, substitution, or swap of neighbors apart.
src/skillevaluator/tier3/eval_core/plugin_signals.py:713
↓ 1 callersFunction_without_atif_token_id_fields
(value: Any)
src/skillevaluator/tier3/harbor/collector.py:1244
↓ 1 callersFunction_without_redirections
A command's words with each redirection and its operand removed, as the shell removes them: ``export -p > f=other.py`` writes a listing to a file
src/skillevaluator/tier3/eval_core/script_invocation.py:2456
↓ 1 callersFunction_without_skill_use
Keep skill use out of the graded text of a starter case that runs unattended. Autopilot evaluates the case right away, and the no-skill baseline
src/skillevaluator/tier3/generate_dataset.py:918
↓ 1 callersFunction_without_url_queries
``text`` without the query of each URL in it: ``scheme://host/path?query`` keeps ``scheme://host/path``. A URL's path ends at its first break (``
src/skillevaluator/validators/url_policy.py:776
↓ 1 callersFunction_wrap_security_error
(exc: SecurePathError)
src/skillevaluator/plugin_manifest.py:375
↓ 1 callersFunction_wrap_segments
Split a row at segment boundaries so it never wraps under the label column.
src/skillevaluator/reporting/console_ui.py:489
↓ 1 callersFunction_wrapped_frontmatter_hooks
The hooks block with every command handler census-wrapped (within the ``MAX_HOOK_*`` bounds).
src/skillevaluator/tier3/plugin_native.py:690
↓ 1 callersFunction_wrapped_stage_runs_stdin
Whether a stage whose wrapper takes an option value (``| sudo -u root bash``, ``| nice -n 10 sh -s``, ``| env -u HISTFILE bash``) runs its standar
src/skillevaluator/plugin_component_risk.py:2278
↓ 1 callersFunction_wrapper_markdown
(name: str, package: PreparedRuntimePackage)
src/skillevaluator/tier3/plugin_prepare.py:639
↓ 1 callersFunction_write
(path: Path, payload: dict)
tests/test_overall_cross_format_consistency.py:24
↓ 1 callersFunction_write
(path: Path, value: object)
tests/reporting/test_dashboard_contract.py:22
↓ 1 callersFunction_write
(path: str, content: str = "export X=1\n")
tests/tier3/test_security_home_paths.py:58
↓ 1 callersFunction_write_arm
(run: Path, arm: str, overall: float)
tests/tier3/test_view_results_feedback.py:57
↓ 1 callersFunction_write_arm
(job, rows)
tests/tier3/test_skill_metric_applicability.py:285
↓ 1 callersFunction_write_arm
One Harbor arm for agent ``opencode`` with one trial of ``case-1``.
tests/evaluation/test_tier3_feedback_chat.py:52
↓ 1 callersFunction_write_attempt_job
(job_dir: Path, *, agent: str, case: str, score: float)
tests/test_harbor_stop_on_pass_concurrency.py:29
↓ 1 callersFunction_write_authentic_pre_status_run
(root: Path, run_id: str, *, score: float = 0.8)
tests/test_tier3_compare.py:406
↓ 1 callersFunction_write_authenticated_pre_status_summary
(run_dir: Path, *, score: float = 0.8)
tests/reporting/test_unified_tier3_report.py:78
↓ 1 callersFunction_write_bounded_failure_artifact
Write a redacted failure record inside the generated JSON envelope.
src/skillevaluator/tier3/harbor/collector.py:1474
↓ 1 callersFunction_write_catalog_atomically_posix
(catalog_path: Path, payload: bytes)
src/skillevaluator/embedding/registry.py:1044
↓ 1 callersFunction_write_catalog_atomically_windows
(catalog_path: Path, payload: bytes)
src/skillevaluator/embedding/registry.py:1180
↓ 1 callersFunction_write_codex_trial
( job_dir: Path, trial: str, trajectory: dict[str, Any], *, session: list[str] | None,
tests/tier3/test_f90_mcp_execution.py:179
↓ 1 callersMethod_write_command_targets
(self, command_name: str, args: str)
src/skillevaluator/tier3/harbor/local_environment.py:1741
↓ 1 callersFunction_write_dataset
Atomically replace a dataset without corrupting an existing file.
src/skillevaluator/tier3/generate_dataset.py:1639
← previousnext →8,201–8,300 of 24,741, ranked by callers