MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 2 callersMethod_run_end
Return where *pattern* ends when matched at *start* (None: no match). *pattern* is a run of units, each a character or a backslash and the
src/skillevaluator/tier3/eval_core/checks.py:4942
↓ 2 callersFunction_run_gitlab_enriched
(tmp_path, extra_env)
tests/ci/test_gateway_benchmark_workflows.py:154
↓ 2 callersFunction_run_grader
()
src/skillevaluator/tier3/harbor/templates/custom_grader_runner.py:328
↓ 2 callersFunction_run_harbor
( *, dataset: Path, agent: str, job_name: str, env_mode: str, model: str, jobs_dir
src/skillevaluator/tier3/harbor/runner.py:4104
↓ 2 callersFunction_run_native_tier
(tmp_path, recomputed)
tests/ci/test_gateway_benchmark_workflows.py:305
↓ 2 callersFunction_run_owned_publication_fields
Return the boundary identity, or a fail-closed mutation marker.
src/skillevaluator/tier3/harbor/runner.py:6088
↓ 2 callersFunction_run_setup
(task: Path, home: Path, logs: Path, *, extra_env: dict | None = None)
tests/tier3/test_native_review_fixes.py:145
↓ 2 callersFunction_run_timestamp
Parse the timestamp prefix shared by current and legacy run names.
src/skillevaluator/tier3/results_location.py:140
↓ 2 callersFunction_run_until_staging
Run the Harbor engine with the provider, prerequisites, staging and trials stubbed out.
tests/tier3/test_judge_policies.py:911
↓ 2 callersFunction_run_verifier
Run the bundled verifier's main() on a no-reference with-skill case.
tests/tier3/test_tool_productivity.py:556
↓ 2 callersFunction_run_vertex_probe_with_deadline
Bound Vertex credential acquisition and probe I/O within a single deadline.
src/skillevaluator/tier3/harbor/runtime_preflight.py:1708
↓ 2 callersFunction_runner_invocation
(argv: list[str])
src/skillevaluator/validators/mcp_static.py:2100
↓ 2 callersFunction_runner_options
A runner's own options, and its first positional argument (the package or command it runs). The runner reads options only up to that argument (or
src/skillevaluator/validators/mcp_static.py:2058
↓ 2 callersFunction_runtime
(root, result)
tests/tier3/test_private_recovered_timeout_review2.py:29
↓ 2 callersMethod_runtime_limit_finding
The v2.13.3 AE1 finding with one runtime_limit reason per ``(limit_seconds, observed_seconds)`` in *runs*.
tests/validators/test_skillspector_reference_diagnostics.py:363
↓ 2 callersFunction_runtime_path
(*parts: str)
src/skillevaluator/utils/scanner_install.py:52
↓ 2 callersFunction_runtime_projection_path_is_ignored
Apply a copytree ignore callback to every component of one source path.
src/skillevaluator/tier3/harbor/adapter.py:907
↓ 2 callersFunction_runtime_source_digest
(root: Path, runtime: RuntimeSourceIdentity)
src/skillevaluator/tier3/plugin_prepare.py:332
↓ 2 callersFunction_safe_environment_kwarg_path
Describe a value location without echoing attacker-controlled nested keys.
src/skillevaluator/tier3/evals_config.py:551
↓ 2 callersFunction_safe_native_path
(path: Path, run_root: Path)
src/skillevaluator/reporting/dashboard_data.py:569
↓ 2 callersFunction_safe_score
(scores: dict[str, float], metric: str)
src/skillevaluator/tier3/commands.py:1808
↓ 2 callersFunction_safe_text
(value: Any, *, max_len: int | None = 2048)
src/skillevaluator/tier3/harbor/collector.py:331
↓ 2 callersFunction_same_entry_alias
Return a lexical alias that opens the skill entry recorded as *recorded_name*. Case- and normalization-insensitive filesystems (the macOS and Win
tests/test_commands.py:754
↓ 2 callersFunction_same_existing_path
(left: Path, right: Path)
src/skillevaluator/publication_identity.py:122
↓ 2 callersFunction_same_existing_path
Recognize filesystem aliases without equating distinct case-sensitive names.
src/skillevaluator/tier3/decoy_commands.py:98
↓ 2 callersFunction_sampled_case_id_diagnostic
Render bounded case-coverage diagnostics while retaining an exact count.
src/skillevaluator/tier3/harbor/collector.py:453
↓ 2 callersFunction_sanitize_reward_metric_surfaces
Omit unsafe metric names without creating redaction aliases.
src/skillevaluator/tier3/harbor/collector.py:4341
↓ 2 callersFunction_sanitize_tier3_agent
(value: dict[str, Any], *, comparison_block_reason: str = "")
src/skillevaluator/reporting/html.py:1011
↓ 2 callersFunction_sanitize_tier3_dimension
( value: dict[str, Any], *, allow_comparison_lift: bool = True, comparison_block_reason: str = "" )
src/skillevaluator/reporting/html.py:632
↓ 2 callersFunction_sanitize_tier3_evaluator_card
( value: dict[str, Any], *, include_evidence_unavailable_default: bool = False, )
src/skillevaluator/reporting/html.py:952
↓ 2 callersFunction_sanitize_tier3_pass_at_k
(value: object)
src/skillevaluator/reporting/html.py:918
↓ 2 callersFunction_sanitize_tier3_trial
Bound the trial fields the template renders. With ``fill_defaults=False`` (baseline rows), optional fields are normalized only when present,
src/skillevaluator/reporting/html.py:1243
↓ 2 callersFunction_sanitized_custom_reward
Keep custom evidence only for validated metric names.
src/skillevaluator/tier3/harbor/templates/custom_grader_runner.py:277
↓ 2 callersFunction_save_unscored_trials
Persist each Harbor trial without a scored reward with a ``failure.json`` record. The record names the attempt's case in ``entry_id``, from the t
src/skillevaluator/tier3/harbor/collector.py:1557
↓ 2 callersFunction_scaled_harbor_timeout
Scale one staged timeout without allowing Harbor to receive infinity.
src/skillevaluator/tier3/harbor/runner.py:3596
↓ 2 callersFunction_scan_images
Scan a skill that names two images with the captured v2.12.0 report; *pattern* replaces each AE1's.
tests/reporting/test_skillspector_reference_summary.py:231
↓ 2 callersFunction_scan_quotes
Read quotes from *pos* to *end*, starting inside the string that opens at *opening* (None: outside one). Return where the next read starts, which
src/skillevaluator/tier3/eval_core/checks.py:5280
↓ 2 callersFunction_scan_sources
One bounded no-follow listing for staging and independent source audits.
src/skillevaluator/tier1/plugin_checks.py:109
↓ 2 callersFunction_scan_stop_reason
Return the diagnostic that says why a scan did not finish, or "". *errors* and *warnings* are the lines that do not repeat a finding. Errors
src/skillevaluator/reporting/html.py:1967
↓ 2 callersFunction_scanner_list
(value)
ci/publish_reports.py:119
↓ 2 callersFunction_scanners_row
The external scanners behind the Tier 1 checks, with their versions or why they did not run.
src/skillevaluator/reporting/console_ui.py:918
↓ 2 callersFunction_schema_errors
( value: Any, schema: Mapping[str, Any], path: str, errors: list[tuple[str, str]], depth: int, budget: lis
src/skillevaluator/tier3/eval_core/plugin_signals.py:3909
↓ 2 callersMethod_scope_before
(self, index: int)
src/skillevaluator/evaluation/dataset_case_removal.py:465
↓ 2 callersFunction_scoped_adc_discovery_environ
Temporarily populate missing ADC discovery env vars in os.environ.
src/skillevaluator/provider_config.py:336
↓ 2 callersFunction_score
(value: object)
ci/report_views.py:113
↓ 2 callersFunction_score
(module, command, **arguments)
tests/tier3/test_review507_merged_regressions.py:16
↓ 2 callersFunction_score_cell
(value: object, *, unavailable: str = "NO SCORE")
src/skillevaluator/tier3/result_display.py:76
↓ 2 callersFunction_score_display_decimal_places
Return one decimal place, or more when rounding would move a score into another verdict band.
src/skillevaluator/reporting/benchmark.py:1922
↓ 2 callersFunction_score_from_line
(line: str)
tests/test_precommit.py:552
↓ 2 callersFunction_score_from_reward
(reward: dict[str, Any])
src/skillevaluator/tier3/harbor/templates/custom_grader_runner.py:214
↓ 2 callersFunction_score_from_text
(text: str)
src/skillevaluator/tier3/harbor/templates/custom_grader_runner.py:218
↓ 2 callersFunction_score_trace
(module, calls: list[dict], *, agent_workdir: str | None = "/workspace")
tests/tier3/test_issue192_security_false_positives.py:59
↓ 2 callersFunction_score_transition
( dimension: dict[str, Any] | None, *, skill_never_activated: bool = False, baseline_not_appli
src/skillevaluator/reporting/benchmark.py:1768
↓ 2 callersFunction_scored
(reward: dict[str, Any], metric: str)
src/skillevaluator/tier3/harbor/lift_basis.py:63
↓ 2 callersFunction_scored
(*scores: float)
tests/test_tier3_case_summary.py:1307
↓ 2 callersFunction_scored_security_details
(reason: str, score: float)
tests/reporting/test_tier3_failure_reasons.py:420
↓ 2 callersFunction_scored_tier3
(verdict: str, *, blocking: bool)
tests/reporting/test_junit_reporter.py:453
↓ 2 callersFunction_script
(output: str, script_id: str)
tests/reporting/test_report_efficiency.py:14
↓ 2 callersFunction_search_path_is_root
(value: str)
src/skillevaluator/tier3/eval_core/checks.py:7921
↓ 2 callersFunction_seatbelt_metadata_ancestors
Return exact existing ancestors needed to traverse to approved roots.
src/skillevaluator/tier3/harbor/local_sandbox.py:726
↓ 2 callersFunction_seatbelt_read_deny_rule
Deny one read operation outside full roots and exact traversal nodes.
src/skillevaluator/tier3/harbor/local_sandbox.py:702
↓ 2 callersFunction_section_heading_lines
Return every matching level-two heading, including untrusted variants.
scripts/ci/check_public_benchmarks.py:1262
↓ 2 callersFunction_section_refs
(section: Any)
src/skillevaluator/plugin_discovery.py:191
↓ 2 callersMethod_secure_windows_download_dir
( self, source_dir: str, target_dir: Path | str, *, excluded_names: se
src/skillevaluator/tier3/harbor/secure_docker_environment.py:2817
↓ 2 callersFunction_security_harness_finding
(source, tool, before)
src/skillevaluator/tier3/eval_core/plugin_security.py:3214
↓ 2 callersFunction_security_reads_content
Handing file names to this command reads the files' content into its output.
src/skillevaluator/tier3/eval_core/plugin_security.py:2631
↓ 2 callersFunction_security_reason
Critical findings first, each message once with its count, at most three messages.
src/skillevaluator/tier3/eval_core/plugin_security.py:3371
↓ 2 callersFunction_security_search_options
How a content search (grep, rg, ag, ack) was called: ``{"letters", "longs", "patterns", "operands", "unknown", "globs"}``. ``unknown`` says its pa
src/skillevaluator/tier3/eval_core/plugin_security.py:2563
↓ 2 callersFunction_security_search_reveals
A content search can print credential content: it prints lines (not only file names or counts), and its pattern can pick out a secret (a secret wo
src/skillevaluator/tier3/eval_core/plugin_security.py:2615
↓ 2 callersFunction_security_tar_members
Member operands of a ``tar`` create, each joined to the ``-C``/``--directory`` in force before it.
src/skillevaluator/tier3/eval_core/plugin_security.py:2515
↓ 2 callersFunction_security_without_redirections
``args`` without redirection operators, their targets, and the fd numbers in front of them.
src/skillevaluator/tier3/eval_core/plugin_security.py:2549
↓ 2 callersFunction_sed_in_place
(io: _CommandIO)
src/skillevaluator/tier3/eval_core/plugin_signals.py:2474
↓ 2 callersFunction_select_artifact
(api, repository: str, run: dict, *, provenance=False)
ci/publish_reports.py:699
↓ 2 callersFunction_selected_host_environment
(names: set[str] | frozenset[str], source: Mapping[str, str])
src/skillevaluator/tier3/harbor/runner.py:2902
↓ 2 callersFunction_selected_path
(root: Path, relative: object)
src/skillevaluator/tier3/decoys.py:126
↓ 2 callersFunction_selected_provider
(environ: Mapping[str, str], variable: str)
src/skillevaluator/provider_config.py:846
↓ 2 callersMethod_send_error
(self, status: int, error_type: str, message: str)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:295
↓ 2 callersFunction_serialize_final_result
Serialize valid JSON without applying the publication size envelope.
src/skillevaluator/tier3/harbor/runner.py:395
↓ 2 callersFunction_serialized_finding_records
(report: Mapping[str, object])
src/skillevaluator/validators/skillspector_scope.py:468
↓ 2 callersFunction_serialized_json_bytes
Return bytes produced by the same JSON settings used for result files.
src/skillevaluator/tier3/harbor/runner.py:696
↓ 2 callersFunction_serialized_payload_size
(payload: dict[str, Any])
src/skillevaluator/evaluation/tier3_report.py:1855
↓ 2 callersFunction_server_bucket
The outcome counts, tool labels and per-tool counts of ``server``, added empty the first time.
src/skillevaluator/tier3/eval_core/plugin_signals.py:4165
↓ 2 callersFunction_server_launch_files
Plugin-root-relative paths an MCP server launch runs or reads, in order, each with whether it is the ``cwd``. ``command`` and ``args`` name plugi
src/skillevaluator/plugin_mcp.py:516
↓ 2 callersFunction_set_codex
(directory: Path, model: str | None, *, status: str = "succeeded")
tests/test_tier3_history.py:232
↓ 2 callersFunction_set_enforcement
(summary: dict, *, mode: str, downgraded: bool)
ci/run_tier1.py:521
↓ 2 callersFunction_set_features
(row: dict[str, Any], *owners: dict[str, Any])
src/skillevaluator/reporting/dashboard_data.py:102
↓ 2 callersFunction_set_reward_case
(run: Path, agent: str, variant: str, trial: str, case: str)
tests/reporting/test_tier3_token_usage_reports.py:163
↓ 2 callersFunction_set_usage
(row: dict[str, Any], usages: list[dict[str, Any]], *, expected: int)
src/skillevaluator/reporting/dashboard_data.py:269
↓ 2 callersFunction_settings_missing_reference_skill
(tmp_path: Path)
tests/validators/test_skillspector_hook_surface.py:253
↓ 2 callersFunction_severities
(findings: list[Any], check: str)
tests/validators/test_plugin_config_risk.py:16
↓ 2 callersFunction_severity_to_level
(severity: str)
src/skillevaluator/reporting/sarif_reporter.py:47
↓ 2 callersFunction_sha256
(text: str)
src/skillevaluator/tier3/decoys.py:59
↓ 2 callersFunction_shannon_entropy
(text: str)
src/skillevaluator/validators/url_policy.py:399
↓ 2 callersFunction_shared_block
(text: str, index: int)
src/skillevaluator/deduplication/intra_skill/llm_analyzer.py:147
↓ 2 callersFunction_shared_case_score
The *dimensions* every arm scored for one case, and each arm's mean over them.
src/skillevaluator/tier3/harbor/stats.py:840
↓ 2 callersFunction_shared_metric_scorer
Apply the trial's score policy to the metrics both arms can compare.
src/skillevaluator/tier3/harbor/collector.py:5364
↓ 2 callersFunction_shared_reference
(index: int)
src/skillevaluator/deduplication/intra_skill/llm_analyzer.py:151
↓ 2 callersFunction_shell_comment_line_numbers
Shell `#` comments, excluding heredoc payloads.
src/skillevaluator/validators/security.py:360
↓ 2 callersFunction_shell_normalized_value
(value: str)
scripts/check_oss_boundary.py:506
↓ 2 callersFunction_shell_program
The program string of ``sh -c '<program>'`` (also ``bash -lc``); ``None`` otherwise.
src/skillevaluator/validators/mcp_static.py:1891
← previousnext →4,501–4,600 of 24,741, ranked by callers