MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 1 callersMethod_frontmatter_rewrites
Staged plugin-arm file -> its source, for each file the plugin arm rewrites to census-wrap frontmatter hooks. The rewrites are the plugin bun
src/skillevaluator/tier3/loose_parts.py:845
↓ 1 callersFunction_frontmatter_root_is_mapping
Return whether the text between the ``---`` lines opens a plain YAML mapping.
src/skillevaluator/deduplication/utils/skill_collector.py:371
↓ 1 callersFunction_frontmatter_skill_name
Read the top-level ``name`` of a SKILL.md the way ``yaml.safe_load`` does. The verifier image gets PyYAML only as a dependency of the RAGAS packa
src/skillevaluator/tier3/harbor/templates/eval.py:4365
↓ 1 callersFunction_gateway_anthropic_agent_credentials
Use a shared gateway unless the operator selected a separate Claude route. A standalone Anthropic key keeps its native endpoint, so a native key
src/skillevaluator/tier3/harbor/runner.py:3019
↓ 1 callersFunction_gating
Return the gate ``validate`` stamped on *result*, or None when it has none.
src/skillevaluator/reporting/junit_reporter.py:241
↓ 1 callersFunction_generate
One random command and the script it is scored against.
scripts/script_invocation_differential.py:1036
↓ 1 callersMethod_generate
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunction_generated_dockerfiles
Every Dockerfile SkillEvaluator writes on the python:3.12-slim base, by its writer.
tests/tier3/test_installed_agent_bootstrap.py:380
↓ 1 callersFunction_generation_input
(kind: str, path: Path, skill_path: Path)
src/skillevaluator/tier3/generate_dataset.py:1660
↓ 1 callersMethod_getActiveElements
(t,e,i,s)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getActiveElements
(t,e,i,s)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getAxisCount
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getLabelBounds
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getLabelSize
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getLegendItemAt
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getOtherScale
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getRegistryForType
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getRotationExtents
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getRuler
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getStackIndex
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getTimestampsForTable
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getUniformDataChanges
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getVisibleDatasetWeightTotal
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getXAxisLabelAlignment
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_getYAxisLabelAlignment
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_get_client
Lazily construct the OpenAI client on first use.
src/skillevaluator/embedding/client.py:154
↓ 1 callersMethod_get_scannable_files
Collect files with scannable extensions from path. Skips Tier 1 artifact directories (``evals/``, ``results/``, ``versions/`` and the
src/skillevaluator/validators/unicode_smuggle.py:533
↓ 1 callersMethod_get_scannable_files
Collect files with scannable extensions from path. Skips evaluation-artifact directories (``evals/``, ``results/``, ``versions/`` and
src/skillevaluator/validators/security.py:6808
↓ 1 callersFunction_get_vertex_access_token
Acquire or refresh a Google Cloud access token in-process for Vertex OpenAPI grading. Attempts discovery via: 1. google.auth.default() (stand
src/skillevaluator/tier3/harbor/templates/eval.py:1352
↓ 1 callersMethod_get_vuln_severity
Extract severity from vulnerability data.
src/skillevaluator/validators/dependencies.py:1032
↓ 1 callersFunction_getaddrinfo_before_deadline
Resolve one host without allowing DNS to hold the caller past its deadline. ``getaddrinfo`` has no portable timeout or cancellation API. One daem
src/skillevaluator/model_catalog.py:247
↓ 1 callersFunction_git_context_files
(root: Path)
src/skillevaluator/tier3/harbor/adapter.py:1239
↓ 1 callersFunction_git_output
Run git in ``cwd`` and return its stripped stdout decoded as UTF-8. Remote URLs and ref names are UTF-8 text, so the locale code page (cp1252
src/skillevaluator/utils/helpers.py:128
↓ 1 callersFunction_github_artifacts
(api, root, run)
ci/feedback.py:326
↓ 1 callersFunction_github_feedback_path
(repo: Path)
src/skillevaluator/_ci_onboarding_all_tiers.py:378
↓ 1 callersFunction_github_trigger_paths
Path filters so only skill or caller changes start the all-tier scan. A pull request outside these paths neither spends runner time nor receives
src/skillevaluator/_ci_onboarding_all_tiers.py:392
↓ 1 callersFunction_github_workflow_inputs
Recognize pinned all-tier callers without treating arbitrary YAML as setup.
src/skillevaluator/_ci_onboarding_all_tiers.py:850
↓ 1 callersFunction_gitlab_minutes
(duration)
tests/ci/test_all_tier_workflows.py:439
↓ 1 callersFunction_gitlab_only_except
GitLab rejects only/except beside rules, including inherited ones; a null value drops an inherited key.
src/skillevaluator/_ci_onboarding_all_tiers.py:938
↓ 1 callersFunction_gitlab_root_needs_merge
A different existing root .gitlab-ci.yml is merged by hand, never replaced.
src/skillevaluator/_ci_onboarding_all_tiers.py:560
↓ 1 callersFunction_gitlab_rules
(document, job, depth=0)
src/skillevaluator/_ci_onboarding_all_tiers.py:905
↓ 1 callersFunction_gitlab_signing_keys
(origin)
ci/feedback.py:697
↓ 1 callersFunction_gitlab_yaml_text
(document)
src/skillevaluator/_ci_onboarding_all_tiers.py:90
↓ 1 callersFunction_global_tools_references
Return ``(start, end)`` spans for possible Codex ``tools`` expressions.
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:616
↓ 1 callersMethod_glue
(self)
src/skillevaluator/tier3/loose_parts.py:1227
↓ 1 callersFunction_glyph
(row: dict[str, Any])
src/skillevaluator/harness/report.py:66
↓ 1 callersFunction_goal_accuracy_budget
Return the configured goal accuracy evidence budget or default.
src/skillevaluator/tier3/eval_core/atif_helpers.py:917
↓ 1 callersFunction_grade
(label: str, call: _Call, call_failures: list[tuple[str, str, str]])
src/skillevaluator/tier3/eval_core/plugin_signals.py:4072
↓ 1 callersFunction_grade_activation_coverage
Declared components exercised, never activated, or whose every activation failed. Entries are ``"<type>:<name>"``. The three outcome lists do not
src/skillevaluator/tier3/eval_core/plugin_signals.py:4688
↓ 1 callersFunction_grade_arguments
Check ``tool_arguments`` rules, and the servers' own input schemas, against every matching call. ``checked``/``passed`` count (call, rule) pairs.
src/skillevaluator/tier3/eval_core/plugin_signals.py:4040
↓ 1 callersFunction_grade_conflict
Each probe passes when ``must_use`` was used and ``must_not_use`` was not tried. ``must_use`` needs a use that did not fail (a failed call or a m
src/skillevaluator/tier3/eval_core/plugin_signals.py:4613
↓ 1 callersFunction_grade_handoff
Verify each ``handoffs[i]`` carried producer output into consumer input (see module docs). A handoff whose producer or consumer names only compon
src/skillevaluator/tier3/eval_core/plugin_signals.py:4542
↓ 1 callersFunction_grade_mcp_calls
Outcome counts across every MCP call (any server), with per-server and per-tool breakdowns. ``success_rate`` is ``succeeded / (succeeded + failed
src/skillevaluator/tier3/eval_core/plugin_signals.py:4175
↓ 1 callersFunction_grade_order
Check ``expected_order`` precedence edges by first use (see :func:`_edge_reason`). Failed calls and failed ``SKILL.md`` reads are not uses. Every
src/skillevaluator/tier3/eval_core/plugin_signals.py:4277
↓ 1 callersFunction_grade_routing
Check 15: precision/recall/F1 of skill, subagent, and command routing. Only routing refs count (typed ``Skill:``/``Agent:``/``Command:`` refs and
src/skillevaluator/tier3/eval_core/plugin_signals.py:3745
↓ 1 callersFunction_grade_tool_selection
Check 22: precision/recall/F1 of MCP and plain tool selection. Only tool refs count (``MCP:`` refs and untyped refs that do not name a declar
src/skillevaluator/tier3/eval_core/plugin_signals.py:3772
↓ 1 callersFunction_grading
(value: Any, config_path: Path)
src/skillevaluator/tier3/evals_config.py:852
↓ 1 callersFunction_ground_truth_ref
(ground_truth: str)
src/skillevaluator/tier3/eval_core/atif_helpers.py:818
↓ 1 callersMethod_group
(self)
src/skillevaluator/plugin_component_risk.py:1330
↓ 1 callersMethod_group_consecutive
Group consecutive invisible characters by column position.
src/skillevaluator/validators/unicode_smuggle.py:242
↓ 1 callersFunction_group_end
(start: int)
src/skillevaluator/tier3/eval_core/checks.py:1820
↓ 1 callersFunction_group_is_gone
()
src/skillevaluator/tier3/nemo_gym/runner.py:641
↓ 1 callersMethod_handleEvent
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_handleMargins
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_handle_no_license
Handle case when no license is detected.
src/skillevaluator/validators/license.py:281
↓ 1 callersMethod_handle_unknown_license
Handle license not in allowlist or blocklist.
src/skillevaluator/validators/license.py:756
↓ 1 callersFunction_handler_for
(config: BridgeConfig)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:637
↓ 1 callersFunction_handoff_artifact_failure
( artifact: str, producer_calls: Sequence[_Call], consumer_calls: Sequence[_Call], root_cwd: str | None )
src/skillevaluator/tier3/eval_core/plugin_signals.py:4529
↓ 1 callersFunction_handoff_calls
``(producer calls, consumer calls)``: calls matching a side directly or running in its window. A window opens at a call that switched to a declar
src/skillevaluator/tier3/eval_core/plugin_signals.py:3504
↓ 1 callersFunction_handoff_value_failure
( value: str, producer_calls: Sequence[_Call], consumer_calls: Sequence[_Call], prompts: Sequence[str] )
src/skillevaluator/tier3/eval_core/plugin_signals.py:4514
↓ 1 callersFunction_harbor_environment_kwargs_from_installed_source
Read the pinned Harbor sources without importing optional backend SDKs.
tests/test_oss_packaging.py:206
↓ 1 callersFunction_harbor_job_owner
(job_name: str, agents: list[str])
src/skillevaluator/tier3/case_summary.py:1595
↓ 1 callersFunction_harbor_mcp_headers
Use Harbor's braced placeholders after validating the original headers.
src/skillevaluator/tier3/harbor/adapter.py:2131
↓ 1 callersFunction_harbor_multi_step_reward_strategy
Read the task's strategy when its staged configuration is still available.
src/skillevaluator/tier3/harbor/collector.py:5482
↓ 1 callersFunction_harbor_step_segments
Split a merged multi-step trajectory at each change of ``extra.harbor_step_name``.
src/skillevaluator/tier3/harbor/trajectory_timing.py:338
↓ 1 callersFunction_harbor_supports_yes
The supported Harbor CLI accepts ``--yes`` for non-interactive runs.
src/skillevaluator/tier3/harbor/runner.py:1568
↓ 1 callersFunction_harbor_usage
Finite recorded counters, summing complete native steps without a root mirror.
src/skillevaluator/tier3/plugin_evidence.py:124
↓ 1 callersFunction_harbor_viewer_evidence_links
(rewards: list[dict[str, Any]])
src/skillevaluator/tier3/harbor/report.py:1065
↓ 1 callersFunction_harbor_viewer_summary
(trials: list[dict[str, Any]])
src/skillevaluator/evaluation/tier3_report.py:3982
↓ 1 callersFunction_harness_flagged
True when the advisory harness check found a harness that drops or degrades a skill.
src/skillevaluator/reporting/console_ui.py:1127
↓ 1 callersFunction_harness_sources
Partition a materialized native multi-step trajectory by its authorized sources. A physical steps layout makes the materializer require authorita
src/skillevaluator/tier3/plugin_trace_artifacts.py:445
↓ 1 callersFunction_has_accelerator_resource
Return True if a V1ResourceRequirements requests or limits GPUs or TPUs.
src/skillevaluator/tier3/harbor/gke_environment.py:201
↓ 1 callersFunction_has_api_documentation
(content: str)
src/skillevaluator/validators/quality_score.py:193
↓ 1 callersFunction_has_codex_tool_reference
Detect executable global Codex-tool references without evaluating JavaScript.
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:743
↓ 1 callersFunction_has_decision_text_alias
Return whether visible text contains a canonical or confusable decision phrase. The shared publication skeleton is intentionally limited to the a
scripts/ci/check_public_benchmarks.py:1776
↓ 1 callersMethod_has_executable_directory
(skill_path: Path)
src/skillevaluator/validators/quality_score.py:296
↓ 1 callersFunction_has_font_signature
Return whether *path* starts with a TrueType, OpenType, collection, or WOFF signature.
src/skillevaluator/validators/skillspector_scope.py:857
↓ 1 callersFunction_has_input_validation
Heuristic check for input validation patterns.
src/skillevaluator/validators/script_lint.py:78
↓ 1 callersFunction_has_literal_secret
Check if argument contains a literal secret matching secret patterns.
src/skillevaluator/tier3/eval_core/checks.py:1925
↓ 1 callersFunction_has_magic_numbers
Check if the AST contains magic number constants.
src/skillevaluator/validators/script_lint.py:61
↓ 1 callersFunction_has_manifest_entry
Return whether *target* is, or directly holds, an artifact manifest entry in any letter case. The manifests are ``SKILL.md``, ``workflow-rules.md
src/skillevaluator/tier1/commands.py:184
↓ 1 callersFunction_has_mapped_evaluator
Return true when the payload contains any primary mapped evaluator key.
src/skillevaluator/evaluation/dimension_judge.py:270
↓ 1 callersFunction_has_native_timing
(steps: list[dict[str, Any]])
src/skillevaluator/tier3/harbor/trajectory_timing.py:104
↓ 1 callersFunction_has_negative_control_semantics
(text: str)
src/skillevaluator/evaluation/insights_judge.py:498
↓ 1 callersFunction_has_nested_markdown_reference
Return whether a reference document links to another local Markdown document.
src/skillevaluator/validators/quality_score.py:234
↓ 1 callersFunction_has_non_js_space
Whether the code, outside string literals and the pragma line, holds a `_NON_JS_SPACE` character.
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:289
↓ 1 callersFunction_has_publication_recommendation
Return whether rendered content recommends publication.
scripts/ci/check_public_benchmarks.py:1793
↓ 1 callersMethod_has_repetition
(self, node: _Node)
src/skillevaluator/plugin_component_risk.py:1451
↓ 1 callersFunction_has_skill_manifest
Return True when *directory* holds a manifest named exactly as in ``SKILL_MANIFEST_VARIANTS``. Names are compared as stored on disk, so the resul
src/skillevaluator/precommit.py:37
↓ 1 callersFunction_has_substantive_guidance
Require authored content beyond Markdown headings and HTML comments.
src/skillevaluator/tier3/generate_dataset.py:131
← previousnext →6,401–6,500 of 24,741, ranked by callers