MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 4 callersFunction_exception_details
Return one Harbor exception type and bounded display reason.
src/skillevaluator/tier3/harbor/collector.py:822
↓ 4 callersFunction_exec_summary
Return the Tier 1 executive summary: verdict, KPIs and Fix first.
tests/reporting/test_reporters.py:800
↓ 4 callersFunction_exit_status
(tc: dict[str, Any])
src/skillevaluator/tier3/eval_core/checks.py:68
↓ 4 callersFunction_expanded
``value`` with the ``$NAME`` / ``${NAME}`` references to known ``variables`` replaced (bounded).
src/skillevaluator/plugin_component_risk.py:2462
↓ 4 callersFunction_expected_records
Each record's JSON text, plus the failed-status line a rejected promise gains.
tests/tier3/test_codex_tool_call_normalization.py:1086
↓ 4 callersFunction_failed_case_findings
An engine result whose with-skill attempts of ``agent`` fail, so the report step lists fixes.
tests/reporting/test_tier3_token_usage_reports.py:586
↓ 4 callersFunction_failed_exit_records
The `pwd` record fails with ``code`` in the given record shape; the second record is clean.
tests/tier3/test_codex_tool_call_normalization.py:1463
↓ 4 callersFunction_failed_gitlab_main
(module)
tests/ci/test_feedback.py:1522
↓ 4 callersFunction_failed_with_skill_agent
An agent whose with-skill arm failed to run, so none of its checks were scored.
tests/test_tier3_result_display.py:239
↓ 4 callersFunction_failure
(case: str = "case-a", attempt: int | None = None, **fields: Any)
tests/tier3/test_baseline_timeouts.py:1464
↓ 4 callersFunction_fake_npm
(monkeypatch, returncode, create_bin)
tests/tier3/test_local_runtime_install.py:37
↓ 4 callersFunction_finding_result
(payload: str)
tests/reporting/test_rich_markup_escaping.py:132
↓ 4 callersMethod_finish
( self, bundle: NativeBundle, source: NativePluginSource, *, setup_lin
src/skillevaluator/tier3/plugin_native.py:1119
↓ 4 callersFunction_finite_reward_number
Convert a Harbor numeric reward without propagating overflow or non-finite values.
src/skillevaluator/tier3/harbor/collector.py:3850
↓ 4 callersMethod_fixes_failure
Failure phrases ("the test case failure", "the case mismatch") name a failure to repair, not the case.
src/skillevaluator/evaluation/dataset_case_removal.py:340
↓ 4 callersFunction_fixture_secret
Build committed fake secrets from pieces so static scanners do not flag them.
tests/tier3/test_secret_redaction_patterns.py:18
↓ 4 callersFunction_flag_values
Return every value passed to one of ``names`` (``--flag v`` or ``--flag=v``).
src/skillevaluator/validators/mcp_static.py:2011
↓ 4 callersFunction_generation
(mode: str)
tests/reporting/test_dataset_generation_provenance.py:20
↓ 4 callersMethod_get
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 4 callersFunction_gitlab_status
(repository, monkeypatch, root_config)
tests/test_ci_onboarding_all_tiers.py:883
↓ 4 callersFunction_gitlab_tier1_consumer
(repository, extra="")
tests/test_ci_onboarding_all_tiers.py:1065
↓ 4 callersFunction_goal_miss
(run_dir: Path, agent: str, trial: str, end_state: str)
tests/test_tier3_case_summary.py:2071
↓ 4 callersFunction_gpt5_config
( *, provider: str = "openai", base_url: str = OPENAI_BASE_URL, model: str = "gpt-5.4-mini", )
tests/inference/test_llm_client.py:46
↓ 4 callersFunction_handoff
(trajectory: dict[str, Any], declared: dict[str, Any] | None = None)
tests/tier3/test_f90_signals_handoff_prompt.py:86
↓ 4 callersFunction_harbor_agent_kwargs
(command: list[str])
tests/test_tier3_public_runtime.py:2856
↓ 4 callersFunction_harbor_launch_cwd
Yield a private, empty working directory for one Harbor process.
src/skillevaluator/tier3/harbor/runner.py:1562
↓ 4 callersFunction_harbor_launch_environment
Return a Harbor child environment that no longer depends on the caller's directory.
src/skillevaluator/tier3/harbor/runner.py:1538
↓ 4 callersFunction_has_judge_reference
Return whether a ground_truth / expected_behavior value gives a judge something to check.
src/skillevaluator/tier3/harbor/templates/eval.py:4068
↓ 4 callersFunction_has_unsafe_control
(value: str)
src/skillevaluator/model_catalog.py:760
↓ 4 callersFunction_hash_path_checked
(path: Path, before: os.stat_result, *, role: str, root_device: int)
src/skillevaluator/tier3/harbor/secure_copy.py:562
↓ 4 callersFunction_heredoc_header
Read the heredocs declared on one line, the first ``<<`` already consumed. Returns the line's command text with each operator and delimiter remov
src/skillevaluator/tier3/eval_core/script_invocation.py:2410
↓ 4 callersFunction_html
(agents: dict[str, dict])
tests/reporting/test_tier3_unavailable_lift_labels.py:51
↓ 4 callersFunction_html_section
(html: str, section_id: str)
tests/reporting/test_tier3_source_validation_reporting.py:84
↓ 4 callersFunction_html_tab
(output: str, tab_id: str)
tests/reporting/test_advisory_skip_verdict.py:52
↓ 4 callersMethod_id_findings
(result)
tests/validators/test_schema.py:1252
↓ 4 callersFunction_identity
()
tests/test_baseline_store.py:57
↓ 4 callersFunction_identity_text_is_publishable
Return whether an identity can safely be used as a generated JSON key.
src/skillevaluator/tier3/harbor/collector.py:348
↓ 4 callersFunction_int
(value: Any)
src/skillevaluator/tier3/eval_core/plugin_signals.py:4843
↓ 4 callersFunction_integer_may_fail
Whether evaluating ``value`` for a ``-i`` name may be an error: not for a number, nor for a name that is unset, empty or holds one (it evaluates t
src/skillevaluator/tier3/eval_core/script_invocation.py:708
↓ 4 callersFunction_invoice_trajectory
()
tests/tier3/test_dataset_grounding.py:53
↓ 4 callersFunction_invoke_with_session
( monkeypatch: pytest.MonkeyPatch, skill: Path, run: Path, session: Tier3FeedbackSession, user_input: str
tests/evaluation/test_tier3_feedback_chat.py:136
↓ 4 callersFunction_is_canonical_openai_base_url
(base_url: str | None)
src/skillevaluator/inference/client.py:76
↓ 4 callersFunction_is_editor_view
A text editor tool call running its ``view`` command (a read).
src/skillevaluator/tier3/eval_core/checks.py:1261
↓ 4 callersFunction_is_jpeg
Whether *data* has the marker structure of a JPEG image. The file must run from SOI to EOI, its last bytes, through well-formed marker segmen
src/skillevaluator/validators/security.py:1385
↓ 4 callersFunction_is_link_or_reparse
Return whether *path* is a symbolic link or Windows reparse point.
src/skillevaluator/deduplication/utils/skill_collector.py:334
↓ 4 callersFunction_is_measurement
A JSON-safe, finite, non-negative counter; bools, strings and NaN are not measurements.
src/skillevaluator/tier3/harbor/resource_usage.py:19
↓ 4 callersFunction_is_name
An argument or property name: a non-empty string of at most ``MAX_ARGUMENT_NAME_CHARS``.
src/skillevaluator/tier3/eval_core/plugin_signals.py:664
↓ 4 callersFunction_is_native_openai_endpoint
(config: ProviderConfig)
src/skillevaluator/inference/client.py:105
↓ 4 callersFunction_is_openai_reasoning_model
(model)
src/skillevaluator/tier3/harbor/templates/eval.py:1276
↓ 4 callersFunction_is_user_skipped
True when the user left this check out with ``--skip``.
src/skillevaluator/reporting/console_ui.py:943
↓ 4 callersFunction_iter_skill_manifests
(root: Path)
src/skillevaluator/tier3/harbor/adapter.py:475
↓ 4 callersFunction_json
(raw)
ci/feedback.py:97
↓ 4 callersFunction_json_report
(output_dir: Path)
tests/test_artifact_applicability.py:86
↓ 4 callersFunction_json_safe_tier3_payload
Return a bounded payload while disclosing any serialization truncation.
src/skillevaluator/reporting/html.py:420
↓ 4 callersFunction_json_type
(value: Any)
src/skillevaluator/tier3/eval_core/plugin_signals.py:3827
↓ 4 callersFunction_judge_not_applicable
Return a scoreless result for a judge that has nothing to judge against.
src/skillevaluator/tier3/harbor/templates/eval.py:4063
↓ 4 callersFunction_judge_retry_policy
Return LLM_JUDGE_RETRY_POLICY: ``legacy`` (default) or ``strict`` (see _call_strict_judge).
src/skillevaluator/tier3/harbor/templates/eval.py:950
↓ 4 callersFunction_legacy_pass_at_k_is_complete
Validate the historical collector's complete pass-at-k structure.
src/skillevaluator/tier3/results_location.py:495
↓ 4 callersMethod_list_dir
List a component folder, nested folders included (see :meth:`PluginRootReader.list_files`). A folder that holds an unsafe entry is HIGH, with
src/skillevaluator/plugin_discovery.py:472
↓ 4 callersMethod_load_json
(self, rel: PurePosixPath, field_name: str)
src/skillevaluator/plugin_discovery.py:535
↓ 4 callersMethod_local_host_env
Return only the host env values local mode is allowed to inherit. Evaluator credentials (``_LIVE_AGENT_KEYS``) are excluded unless the
src/skillevaluator/tier3/harbor/local_environment.py:1576
↓ 4 callersFunction_logical_group_entry_identity_is_consistent
Reject ambiguous multi-row trials whose physical rows claim different cases.
src/skillevaluator/evaluation/tier3_report.py:5133
↓ 4 callersFunction_looks_like_trial_dir
(path: Path)
src/skillevaluator/tier3/harbor/collector.py:1519
↓ 4 callersMethod_lower
(result: ValidationResult, **options: bool)
tests/models/test_result.py:221
↓ 4 callersMethod_main_only_compose_environment
Return main-only names and values that a sidecar client must not inherit.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:1991
↓ 4 callersFunction_manifest
(plugin: Path, skills: list, rules: list | None = None)
tests/validators/test_plugin_dependencies.py:57
↓ 4 callersFunction_matcher_names
(text: str)
src/skillevaluator/plugin_component_risk.py:1570
↓ 4 callersFunction_materialized_trial_trajectory
Select and materialize the trajectory authorized by Harbor topology.
src/skillevaluator/tier3/harbor/collector.py:6343
↓ 4 callersFunction_medium_pii_skill
(tmp_path: Path)
tests/test_cli.py:1273
↓ 4 callersFunction_merged_embedded_subagents
(*collections: Any)
src/skillevaluator/tier3/harbor/collector.py:2224
↓ 4 callersFunction_merged_entries
(manifests: tuple[_TreeManifest, ...])
src/skillevaluator/tier3/harbor/secure_copy.py:1078
↓ 4 callersFunction_messages
(skill)
tests/validators/test_quality_score.py:2149
↓ 4 callersFunction_mixed_info
One case scored by a fallback judge in the with-skill arm, one by matching judges.
tests/test_judge_lift_integrity.py:552
↓ 4 callersFunction_mock_skillspector
(mock_tools, report_for_root)
tests/validators/test_skillspector_scope.py:1640
↓ 4 callersFunction_model_endpoint
Return ``(model, base_url, api_key)`` for Gym's OpenAI-compatible policy model server.
src/skillevaluator/tier3/nemo_gym/runner.py:309
↓ 4 callersFunction_model_leaf
Return a normalized model ID for capability checks only.
src/skillevaluator/provider_config.py:147
↓ 4 callersFunction_model_leaf
(model)
src/skillevaluator/tier3/harbor/templates/eval.py:1257
↓ 4 callersFunction_mutate_native_task_toml
Mutate a staged native task structurally and revalidate with Harbor.
src/skillevaluator/tier3/harbor/adapter.py:5602
↓ 4 callersFunction_named
(name: str, steps: list[dict[str, Any]])
tests/tier3/test_trajectory_timing.py:375
↓ 4 callersFunction_names_by_kind
Declared skill, subagent, and command names per type (see :func:`_ref_unavailable`). Empty when the arm carries every type, since then no ref is
src/skillevaluator/tier3/eval_core/plugin_signals.py:3602
↓ 4 callersFunction_native_entry_id
Resolve and validate the canonical case ID for a native Harbor task directory.
src/skillevaluator/tier3/harbor/adapter.py:4735
↓ 4 callersFunction_native_verifier_passes
Resolve every verify pass with Harbor's own separate-verifier precedence.
src/skillevaluator/tier3/harbor/adapter.py:5333
↓ 4 callersFunction_no_change
(directory)
tests/ci/test_smoke_image.py:15
↓ 4 callersFunction_normalize_anthropic_base_url
(value: str, *, variable: str)
src/skillevaluator/provider_config.py:701
↓ 4 callersFunction_normalized_match_text
Normalize ordinary Unicode and Markdown spellings, without mapping homoglyphs.
src/skillevaluator/tier3/evals_spec.py:810
↓ 4 callersMethod_note_references
(self, value: Any, *, exact_operand: bool = False)
src/skillevaluator/tier3/loose_parts.py:1098
↓ 4 callersFunction_number
(value: object)
src/skillevaluator/reporting/benchmark.py:1749
↓ 4 callersFunction_number
(value: Any)
src/skillevaluator/reporting/dashboard_data.py:65
↓ 4 callersFunction_observe_preflight
(monkeypatch: pytest.MonkeyPatch)
tests/test_tier3_config_key_parity.py:283
↓ 4 callersFunction_offer_scanner_install
Before Tier 1, install the pinned, checksum-verified SkillSpector and Gitleaks when missing.
src/skillevaluator/cli.py:945
↓ 4 callersFunction_one
(collection: McpCollection)
tests/validators/test_openclaw_mcp.py:40
↓ 4 callersFunction_open_named_node
(parent_descriptor: int, name: str, *, kind: str)
src/skillevaluator/tier3/harbor/secure_copy.py:818
↓ 4 callersFunction_overall_kpi_values
(output: str)
tests/test_harbor_html_report_failures.py:56
↓ 4 callersFunction_overflow_note
(hidden: int, report: str | None)
tests/reporting/test_tier3_source_validation_reporting.py:61
↓ 4 callersFunction_package
(root: Path, files: dict)
tests/tier3/test_private_runtime_regressions.py:43
↓ 4 callersFunction_parse
(tmp_path, frontmatter: str)
tests/tier3/test_generate_dataset_results.py:561
↓ 4 callersFunction_parse_skill
Parse SKILL.md for name, description, scripts, interactive hints, and developer eval guidance. Developer eval guidance is loaded from (in priorit
src/skillevaluator/tier3/generate_dataset.py:139
↓ 4 callersFunction_pass_rate_delta
Return the legacy delta contract derived from persisted rounded rates.
src/skillevaluator/tier3/harbor/collector.py:4975
↓ 4 callersFunction_passing_dimensions
(**overrides: object)
tests/evaluation/test_tier3_benchmark_contract.py:369
↓ 4 callersMethod_path_map
(self)
src/skillevaluator/tier3/harbor/local_environment.py:1939
← previousnext →2,101–2,200 of 24,741, ranked by callers