MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 1 callersMethod_render_incomplete
Render missing scanner evidence without presenting a failure as a pass.
src/skillevaluator/reporting/markdown.py:688
↓ 1 callersFunction_render_incomplete_execution
Show execution evidence once, without implying an unscored skill failed.
src/skillevaluator/tier3/result_display.py:400
↓ 1 callersMethod_render_metadata
( self, lines: list[str], results: list[ValidationResult], ae: dict[str, Any]
src/skillevaluator/reporting/benchmark.py:393
↓ 1 callersMethod_render_methodology
( lines: list[str], ae: dict[str, Any] | None, private_labels: tuple[str, ...], )
src/skillevaluator/reporting/benchmark.py:960
↓ 1 callersFunction_render_noop
(reports: Path, summary: dict, context: dict)
ci/report_views.py:380
↓ 1 callersMethod_render_plan
(self, plan: Tier3RunPlan)
src/skillevaluator/tier3/harbor/progress.py:314
↓ 1 callersMethod_render_publication_recommendation
( self, lines: list[str], results: list[ValidationResult], )
src/skillevaluator/reporting/benchmark.py:377
↓ 1 callersMethod_render_reference_summary
List how many referenced artifacts SkillSpector did not completely inspect in a failed result.
src/skillevaluator/reporting/markdown.py:679
↓ 1 callersMethod_render_report_purpose
(lines: list[str])
src/skillevaluator/reporting/benchmark.py:556
↓ 1 callersMethod_render_success
Render success details.
src/skillevaluator/reporting/markdown.py:606
↓ 1 callersMethod_render_tier_status
( self, lines: list[str], results: list[ValidationResult], ae: dict[str, Any]
src/skillevaluator/reporting/benchmark.py:821
↓ 1 callersFunction_render_token_usage
Print the per-agent token sums that the report step recorded: measured, not scored. The pointer to the case rows names only the reports the run r
src/skillevaluator/tier3/result_display.py:1130
↓ 1 callersFunction_render_user_prompt
Render every member; texts in ``shared`` are listed once and referenced.
src/skillevaluator/deduplication/intra_skill/llm_analyzer.py:169
↓ 1 callersFunction_rendered
(rows)
tests/test_artifact_applicability.py:66
↓ 1 callersFunction_repeated
Return a function that writes *head*, *unit* repeated and *tail* in about the length it is given.
tests/tier3/test_security_finding_evidence.py:2660
↓ 1 callersFunction_replace_evaluator_tests_dir
Replace the evaluator-owned verifier payload inside a staged task.
src/skillevaluator/tier3/harbor/adapter.py:2413
↓ 1 callersFunction_replace_retired_product_tokens
Replace retired identity tokens even when Unicode marks split the spelling.
src/skillevaluator/reporting/benchmark.py:2310
↓ 1 callersMethod_replace_shell_path
Replace a container path while preserving shell token boundaries.
src/skillevaluator/tier3/harbor/local_environment.py:1979
↓ 1 callersFunction_replace_single_from_image
Replace one Dockerfile stage image while preserving options and its alias.
src/skillevaluator/tier3/harbor/adapter.py:2648
↓ 1 callersMethod_replacement_follows
(self, end: int)
src/skillevaluator/evaluation/dataset_case_removal.py:479
↓ 1 callersFunction_repo_path_is_skill_evals
Identify evaluator-only roots without dropping unrelated nested ``evals``.
src/skillevaluator/tier3/harbor/adapter.py:1075
↓ 1 callersFunction_report
(run: ToolResult, allowed_codes: tuple[int, ...])
src/skillevaluator/plugin_dependency_audit.py:470
↓ 1 callersMethod_report_bandit_issue
Report a single Bandit finding.
src/skillevaluator/validators/code_risk.py:270
↓ 1 callersFunction_report_base
(value: str)
ci/tier_manifest.py:334
↓ 1 callersFunction_report_link
A downloadable archive is one artifact, never a member URL directory.
ci/comment_summary.py:41
↓ 1 callersFunction_report_options
(func)
src/skillevaluator/cli.py:488
↓ 1 callersMethod_report_path
Return the waiver's file as the PII findings of a scan of ``target`` name it. That is relative to the covered unit, with the ``[skill] `` pre
src/skillevaluator/validators/pii_waivers.py:301
↓ 1 callersMethod_report_semgrep_finding
Report a single Semgrep finding.
src/skillevaluator/validators/code_risk.py:474
↓ 1 callersMethod_report_semgrep_parse_errors
Report files Semgrep could not fully parse as a partial scan. The message names up to ``_SEMGREP_PARSE_ERROR_PATH_LIMIT`` files with the line
src/skillevaluator/validators/code_risk.py:447
↓ 1 callersMethod_report_unverified_hooks
Fail the gate for each manifest whose hooks SkillSpector could not model, and return the manifests. *hook_rows* come from :func:`_skillspecto
src/skillevaluator/validators/security.py:5286
↓ 1 callersMethod_report_vulnerability
Report one pip-audit advisory as a ``DEPENDENCY`` finding at its declaration. ``declared`` is ``None`` only for a caller that audited pins it
src/skillevaluator/validators/dependencies.py:851
↓ 1 callersMethod_reported_advisories
Return ``(package, version, ID or alias)`` keys of the pip-audit advisories in ``result``.
src/skillevaluator/validators/dependencies.py:747
↓ 1 callersMethod_reported_value
Return the value a PII finding identifies for ``match``, before any display redaction. A finding's ``evidence_sha256`` and the waiver check b
src/skillevaluator/validators/security.py:6689
↓ 1 callersFunction_repository_paths
(root: Path)
scripts/check_oss_boundary.py:796
↓ 1 callersFunction_request_build
(config: BridgeConfig, payload: dict[str, Any])
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:355
↓ 1 callersMethod_request_max_tokens
Return the output-token budget for a request to the resolved target. An explicit ``max_tokens`` skips this default adjustment. Provider and
src/skillevaluator/inference/client.py:600
↓ 1 callersMethod_request_rubric
(self, system_prompt: str, user_prompt: str)
src/skillevaluator/validators/rubric_eval.py:405
↓ 1 callersMethod_require_pinned_skillspector
Keep automatic SkillSpector resolution on the supported pinned release.
src/skillevaluator/utils/tool_runner.py:214
↓ 1 callersMethod_require_source
(self)
src/skillevaluator/tier3/loose_parts.py:535
↓ 1 callersFunction_require_unit_interval
Require a JSON number (not a bool or a string) that is finite and within [0, 1].
src/skillevaluator/inference/types.py:70
↓ 1 callersFunction_required_attempts
(flags: Sequence[bool], *, n_attempts: int, stop_on_pass: bool)
src/skillevaluator/tier3/harbor/stats.py:1463
↓ 1 callersFunction_required_check_comparison
(info: dict[str, Any])
src/skillevaluator/evaluation/tier3_report.py:3510
↓ 1 callersMethod_required_compose_model_environment_names
Return available, non-infrastructure names required by the model.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:2219
↓ 1 callersFunction_requirements
(case: dict[str, Any])
src/skillevaluator/evaluation/tier3_expectations.py:42
↓ 1 callersFunction_rerun_hint
Reconstruct the user's actual command for the FAIL panel's rerun line. A bare ``skillevaluator validate <path>`` would drop the flags that shaped
src/skillevaluator/cli.py:1955
↓ 1 callersFunction_resample_means
The mean of each case resample, in draw order, from a ``random.Random(seed)`` stream. ``rng.choices`` picks every index from the seed and ``len(d
src/skillevaluator/tier3/harbor/stats.py:568
↓ 1 callersMethod_resetElements
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunction_reset_agent_generated_outputs
Clear only collector/report-owned artifacts before rebuilding an agent result.
src/skillevaluator/tier3/harbor/collector.py:306
↓ 1 callersFunction_resolve
(command: str)
tests/test_skillevaluator_skill.py:34
↓ 1 callersFunction_resolve_acceptable_skills
(entry, expected_skill=None)
src/skillevaluator/tier3/harbor/templates/eval.py:2854
↓ 1 callersMethod_resolve_auth_env
Resolve Claude Code authentication environment including Vertex AI variables.
src/skillevaluator/tier3/harbor/local_agents.py:220
↓ 1 callersFunction_resolve_backend_before_deadline
Resolve one backend host without allowing DNS to outlive the request deadline.
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:545
↓ 1 callersMethod_resolve_content_type
Determine content type from explicit flag or auto-detection.
src/skillevaluator/validators/similarity.py:227
↓ 1 callersFunction_resolve_exec_records
Give each call of a record wrapper its own record, or leave the wrapper unsupported.
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:1425
↓ 1 callersMethod_resolve_frontmatter_and_files
Reconcile a concrete frontmatter license with every license file.
src/skillevaluator/validators/license.py:513
↓ 1 callersFunction_resolve_headers
Return ``(headers to send, unset variables, withheld variables, sent variables)``. A header whose ``${VAR}``/``$VAR`` references all name variabl
src/skillevaluator/tier3/mcp_proof.py:250
↓ 1 callersFunction_resolve_manifest_reads
``(read, failed)`` for each read that ran, in order; a read skipped by a failed ``&&`` part is dropped.
src/skillevaluator/tier3/eval_core/plugin_signals.py:2706
↓ 1 callersFunction_resolve_relative_kubeconfig_path
Resolve a relative kubeconfig path against its source file directory.
src/skillevaluator/tier3/harbor/kubeconfig.py:29
↓ 1 callersMethod_resolve_task_runtime_env
Extract allowed runtime environment from task.toml if available.
src/skillevaluator/tier3/harbor/local_agents.py:297
↓ 1 callersFunction_resolve_trial_name_entry_id
(trial_name_val: str)
src/skillevaluator/tier3/harbor/collector.py:3664
↓ 1 callersFunction_resolve_url
(provider: str)
src/skillevaluator/tier3/eval_core/llm_judge.py:104
↓ 1 callersFunction_resolve_url_references
The URL to check for an MCP URL that uses environment expansion, or ``None`` when it cannot be checked. Claude Code (and, leniently, Cursor and A
src/skillevaluator/validators/mcp_static.py:914
↓ 1 callersFunction_resolve_within
Run ``resolver`` in a daemon worker thread and wait at most ``timeout`` seconds. ``getaddrinfo`` cannot be interrupted, so a black-holed name wou
src/skillevaluator/tier3/mcp_proof.py:317
↓ 1 callersFunction_resolved_agent_model
(result: Mapping[str, Any], agent: str, data: Mapping[str, Any])
src/skillevaluator/tier3/result_display.py:536
↓ 1 callersFunction_resolved_colors
Foreground and background the cascade gives an element that `matching` selectors select.
tests/reporting/test_report_redesign.py:560
↓ 1 callersFunction_resolved_dimension_mapping
Pick the primary evaluator mapping, or a legacy fallback when absent.
src/skillevaluator/evaluation/dimension_judge.py:275
↓ 1 callersFunction_resolved_existing_path
(path: str | Path)
src/skillevaluator/tier3/harbor/local_runtime.py:132
↓ 1 callersFunction_resolved_projection_task_source
( requested_task_source: str | None, evals_config: dict[str, Any], *, dataset_path: Path | Non
src/skillevaluator/tier3/harbor/adapter.py:6407
↓ 1 callersFunction_resolved_skill_name
(result: Mapping[str, Any])
src/skillevaluator/tier3/result_display.py:545
↓ 1 callersFunction_resources
(value: Any, config_path: Path)
src/skillevaluator/tier3/evals_config.py:761
↓ 1 callersFunction_responses_custom_tool_call_to_chat
( item: dict[str, Any], namespace_tools: dict[str, _NamespaceToolTarget] | None = None )
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1626
↓ 1 callersFunction_responses_function_call_to_chat
( item: dict[str, Any], namespace_tools: dict[str, _NamespaceToolTarget] | None = None )
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1608
↓ 1 callersFunction_responses_messages
( input_value: Any, namespace_tools: dict[str, _NamespaceToolTarget] | None = None )
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1565
↓ 1 callersFunction_responses_namespace_tool_to_chat
( tool: dict[str, Any], namespace_tools: dict[str, _NamespaceToolTarget], top_level_names: set[str
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1727
↓ 1 callersFunction_responses_response
(response_id: str, model: str, status: str, output: list[dict[str, Any]])
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1511
↓ 1 callersFunction_responses_tool_choice_to_chat
(choice: Any, namespace_tools: dict[str, _NamespaceToolTarget] | None = None)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1474
↓ 1 callersFunction_responses_tools_to_chat
( tools: list[Any], )
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1700
↓ 1 callersFunction_restates
True when the eval prompt pastes in a long run of the skill description instead of posing a task.
src/skillevaluator/tier3/case_summary.py:1368
↓ 1 callersFunction_restore_failure_reason_text
Put deferred failure reasons back while ``room`` bytes remain; return how many stay out.
src/skillevaluator/reporting/html.py:1696
↓ 1 callersFunction_result
(method: str, params: dict[str, Any])
tests/tier3/test_mcp_proof.py:109
↓ 1 callersFunction_result_agent_eval_run_id_issue
Reject a persisted outer Tier 3 run claim that contradicts its payload.
src/skillevaluator/reporting/base.py:1988
↓ 1 callersFunction_result_artifact_reference
Describe one regular contained artifact by stable relative path and digest.
src/skillevaluator/tier3/harbor/runner.py:403
↓ 1 callersFunction_result_flagged
A result's structured outcome: ``True`` failed, ``False`` succeeded, ``None`` no signal. A failure flag is any ``is_error``/``isError`` (top leve
src/skillevaluator/tier3/eval_core/plugin_signals.py:1909
↓ 1 callersFunction_result_from_finding
( finding: Finding, validator_name: str, workspace_root: Path | None, scan_root: Path | None =
src/skillevaluator/reporting/sarif_reporter.py:223
↓ 1 callersFunction_result_publication_target_conflict_markers
Return bounded direct and nested conflict markers for one result.
src/skillevaluator/reporting/base.py:902
↓ 1 callersFunction_result_task_path_entry_id
Return (metadata_entry_id, dir_name) from a Harbor task path when available.
src/skillevaluator/tier3/harbor/collector.py:4150
↓ 1 callersFunction_results_for_call
Results attributable to ``call_id``; id matches win, ambiguity stays unknown.
src/skillevaluator/tier3/eval_core/plugin_signals.py:1994
↓ 1 callersMethod_resyncElements
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunction_retained_diagnostics
Members of one real worker-diagnostics tree for a failed Harbor run.
tests/ci/test_feedback.py:1154
↓ 1 callersFunction_retained_run
(path: Path, status: str, *, persist_census: bool)
tests/reporting/test_dashboard_activation_regressions.py:134
↓ 1 callersFunction_retained_run
(run: Path, *, attempts: int)
tests/reporting/test_dashboard_contract.py:27
↓ 1 callersMethod_retained_skillspector_risk_score
Score *issues* with SkillSpector's formula, one finding per serialized record. Use only when the report serializes every finding it counted.
src/skillevaluator/validators/security.py:4875
↓ 1 callersFunction_returned_outcome
Return the outcome of a tool that returned: only its own exit code says whether it succeeded.
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:1306
↓ 1 callersFunction_revalidate_components
( output_path: Path, snapshots: list[tuple[Path, os.stat_result]], )
src/skillevaluator/reporting/base.py:342
↓ 1 callersFunction_revalidate_finished_posix_directory
Require a finished directory to be unchanged and still bound to its declared name.
src/skillevaluator/utils/secure_fs.py:1281
↓ 1 callersFunction_reward
(case, *, accuracy, baseline=False)
tests/reporting/test_applicability_saved_runs.py:26
↓ 1 callersFunction_reward
(scores: dict[str, float], trial_id: str, judge: str)
tests/reporting/test_tier3_unavailable_lift_labels.py:20
↓ 1 callersFunction_reward
()
tests/tier3/test_report_renders_refs.py:7
↓ 1 callersFunction_reward_canary
(reward: Mapping[str, Any])
src/skillevaluator/tier3/eval_core/runtime_evidence.py:241
↓ 1 callersFunction_reward_entries
Distinct evidence entries of the score-impacting ``finding_type`` findings of one trial.
src/skillevaluator/tier3/eval_core/runtime_evidence.py:247
← previousnext →7,401–7,500 of 24,741, ranked by callers