MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 5 callersFunction_resolve_runtime_path
(path: Path, *, strict: bool = True)
src/skillevaluator/tier3/harbor/local_runtime.py:224
↓ 5 callersMethod_resolved_config
(self)
src/skillevaluator/embedding/client.py:132
↓ 5 callersFunction_result_publication_target
(result: ValidationResult)
src/skillevaluator/reporting/base.py:827
↓ 5 callersFunction_results
(*, skill_like=True, omitted=None, unscanned=None)
tests/reporting/test_retained_publication_scope.py:15
↓ 5 callersFunction_reward_policy
(reward: Mapping[str, Any], *, logical: bool = False)
src/skillevaluator/tier3/harbor/stats.py:128
↓ 5 callersMethod_rewrite_command
(self, command: str)
src/skillevaluator/tier3/harbor/local_environment.py:1952
↓ 5 callersFunction_row
(table: str, condition: str)
tests/reporting/test_tier3_baseline_trial_views.py:179
↓ 5 callersMethod_run_bandit
Run Bandit security linter on Python code.
src/skillevaluator/validators/code_risk.py:168
↓ 5 callersFunction_runtime_copy_lines
( agent_config_lines: list[str], include_input: bool, include_repo: bool = False, include_repo
src/skillevaluator/tier3/harbor/adapter.py:3861
↓ 5 callersFunction_safe_evaluation_errors
Normalize verifier diagnostics before persisting or displaying them.
src/skillevaluator/tier3/harbor/collector.py:488
↓ 5 callersFunction_safe_trial_path_component
Return a portable single path component or an empty string.
src/skillevaluator/tier3/harbor/collector.py:5857
↓ 5 callersFunction_sanitize_tier3_display_payload
Normalize template-facing types after merging canonical and fallback data.
src/skillevaluator/reporting/html.py:1333
↓ 5 callersMethod_scan_with_skillspector_reports
(skill_dir: Path, reports: list[dict], *, use_llm: bool)
tests/validators/test_security.py:5810
↓ 5 callersFunction_score
(value: object)
src/skillevaluator/tier3/case_summary.py:588
↓ 5 callersFunction_script_path_matches
Whether one shell argument names the expected script exactly. ``run.py`` matches ``run.py``, ``./run.py`` and ``/skills/demo/run.py``. It doe
src/skillevaluator/tier3/eval_core/script_invocation.py:2075
↓ 5 callersFunction_sensitive_environment_values
Return exact values whose component-aware names mark them sensitive.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:258
↓ 5 callersFunction_sequence
(*results)
tests/tier3/test_plugin_execution.py:34
↓ 5 callersFunction_settings_missing_reference_report
(version: str)
tests/validators/test_skillspector_hook_surface.py:259
↓ 5 callersFunction_settings_skill
(tmp_path: Path)
tests/validators/test_skillspector_hook_surface.py:230
↓ 5 callersFunction_shell_chain
``(command words, joiner before it)`` for each simple command, in order. The joiner is the operator that links a command to the one before it
src/skillevaluator/tier3/eval_core/plugin_signals.py:2370
↓ 5 callersMethod_sidecar_operation
Serialize a service while rejecting callback reentry before waiting.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:2327
↓ 5 callersFunction_signals
(trajectory: dict[str, Any], **kwargs: Any)
tests/tier3/test_f90_mcp_execution.py:60
↓ 5 callersFunction_skill
(root: Path, *, eval_name: str = "eval")
tests/tier3/test_runtime_eval_directories.py:30
↓ 5 callersFunction_skill
(tmp_path: Path)
tests/tier3/test_tier3_source.py:12
↓ 5 callersFunction_skill_with_outside
(tmp_path: Path)
tests/inference/test_finding_verifier.py:11
↓ 5 callersFunction_skillspector_incomplete_analysis_error
Describe a partial SkillSpector analysis with example gaps and, for runtime limits, a remedy.
src/skillevaluator/validators/security.py:1241
↓ 5 callersFunction_skillspector_name_keys
Case-folded forms of *names*, as written and in NFC, for :func:`_skillspector_reference_line_avoids`.
src/skillevaluator/validators/security.py:1944
↓ 5 callersFunction_skillspector_process_env
(environ: Mapping[str, str] | None = None)
src/skillevaluator/validators/security.py:671
↓ 5 callersFunction_skillspector_reference_line_avoids
Whether the line of a missing-reference *row* names no entry whose name key is in *keys*. The line is taken from *source* as SkillSpector numbers
src/skillevaluator/validators/security.py:2326
↓ 5 callersFunction_skillspector_unscanned_names
Every path part of an uninspected entry, or ``None`` when the bounded walk fails. A directory that holds an uninspected entry is named even when
src/skillevaluator/validators/security.py:1934
↓ 5 callersFunction_skillspector_without_hook_surface
A ``partial`` report's gaps without the rows for hooks SkillSpector could not model, else ``None``. Returns the hook rows (:func:`_skillspector_l
src/skillevaluator/validators/security.py:2819
↓ 5 callersFunction_stage_repo_context
Stage repo files referenced by SKILL.md, or the full repo when requested.
src/skillevaluator/tier3/harbor/adapter.py:1296
↓ 5 callersFunction_standard_reward_metrics
Return required standard metrics, preserving explicit child metric-set declarations.
src/skillevaluator/tier3/harbor/collector.py:3212
↓ 5 callersFunction_succeeded
(calls)
tests/tier3/test_opencode_exit_status.py:37
↓ 5 callersFunction_supports_custom_temperature
Return whether ``model`` accepts a non-default temperature value.
src/skillevaluator/provider_config.py:178
↓ 5 callersMethod_sync
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callersFunction_t
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callersMethod_test_discovery_detail
(result)
tests/validators/test_hygiene.py:17
↓ 5 callersFunction_text
(fragment: str)
tests/test_judge_lift_integrity.py:370
↓ 5 callersFunction_trajectory
(messages: list)
tests/tier3/test_atif_message_content.py:16
↓ 5 callersFunction_trajectory_with_long_final_response
Build a synthetic trajectory with a long agent response. ``middle`` places ``_LATE_MARKER_`` far enough into the answer that a cap below it d
tests/tier3/test_behavior_evidence.py:775
↓ 5 callersFunction_unavailable_kinds
Component types this arm cannot carry: declared by the plugin but not staged here, or not on this harness.
src/skillevaluator/tier3/eval_core/plugin_signals.py:3574
↓ 5 callersFunction_unknown_keys
Return the keys outside ``allowed``; a non-string key such as ``1``, ``yes`` or ``~`` is an error.
src/skillevaluator/tier3/evals_config.py:233
↓ 5 callersFunction_unscanned_folder
The first folder of a plugin-root-relative path that Tier 1 whole-tree scans skip, or ``None``. The security, secret, and Unicode scans prune eve
src/skillevaluator/plugin_paths.py:370
↓ 5 callersFunction_unverified_usage
(cases: list[dict])
tests/reporting/test_tier3_token_usage_reports.py:1262
↓ 5 callersFunction_upload_app
()
tests/reporting/test_dashboard_app.py:87
↓ 5 callersMethod_uses_vertex_auth
Return whether Claude Code is configured to route through Vertex AI.
src/skillevaluator/tier3/harbor/local_agents.py:192
↓ 5 callersFunction_validate
Scan *skills* under *root* with the captured report; *lines* lists that many reference lines in one AE1.
tests/reporting/test_skillspector_reference_summary.py:54
↓ 5 callersFunction_validate_catalog_string
(value: object, label: str, *, allow_text_whitespace: bool = False)
src/skillevaluator/embedding/registry.py:883
↓ 5 callersFunction_validated_agent_workdir
Return a normalized absolute container workdir or fail closed.
src/skillevaluator/tier3/harbor/adapter.py:3793
↓ 5 callersFunction_validation
(payload)
tests/reporting/test_declarative_coverage.py:119
↓ 5 callersFunction_verifier_install_lines
Install verifier and skill requirements in one resolver transaction. ``import_check=False`` skips the eval.py import check for custom_only gr
src/skillevaluator/tier3/harbor/adapter.py:3586
↓ 5 callersFunction_within
()
tests/reporting/test_evidence_views.py:12
↓ 5 callersFunction_without_token_usage
(payload: dict[str, Any])
tests/reporting/test_tier3_token_usage_reports.py:178
↓ 5 callersFunction_workflow_names
Every workflow on disk, so a new one is covered the day it lands.
tests/test_ci_workflows.py:66
↓ 5 callersFunction_write
Replace a site file atomically; a failed write leaves the old file intact.
ci/publish_reports.py:433
↓ 5 callersMethod_write
(self, tmp_path: Path, card: str)
tests/test_benchmark_source_provenance.py:760
↓ 5 callersFunction_write_json
(path: Path, data: Any)
tests/evaluation/test_tier3_feedback_chat.py:38
↓ 5 callersFunction_write_sized_skill
Write a skill whose SKILL.md is exactly ``characters`` long, so its estimate is characters // 4.
tests/validators/test_quality_score.py:1697
↓ 5 callersMethodadd
(self, key: str, *cells: object)
src/skillevaluator/reporting/plugin_runtime.py:69
↓ 5 callersMethodadd_flag
Record a risk flag; each flag is listed (and counted in the summary) once per handler.
src/skillevaluator/plugin_component_risk.py:1129
↓ 5 callersFunctionagent_eval_publication_target
Return the duplicated Tier 3 target claim only when both copies agree.
src/skillevaluator/reporting/base.py:913
↓ 5 callersFunctionagent_eval_result_from_run
Build an advisory ``AGENT_EVAL`` result from the latest on-disk Harbor run. Returns ``None`` when no usable run directory or agent data can be fo
src/skillevaluator/evaluation/tier3_report.py:475
↓ 5 callersFunctionagent_model_overrides
Parse bounded comma/newline native agent model choices without rewriting IDs.
ci/model_config.py:158
↓ 5 callersFunctionagent_result
()
tests/reporting/test_report_navigation.py:60
↓ 5 callersFunctionalign_verdict
Make explicitly mutated fixtures internally consistent; mismatch tests opt out.
tests/ci/test_report_views.py:71
↓ 5 callersFunctionallowed_tools_flag_issues
``--allowedTools`` passed to an agent CLI with a grant that runs any command (or ``*``), HIGH each. The launched agent then runs any shell comman
src/skillevaluator/plugin_permissions.py:291
↓ 5 callersMethodallows
Whether an entry covers a non-public endpoint, by its host name or by a network holding its address.
src/skillevaluator/validators/mcp_static.py:3011
↓ 5 callersFunctionanalyze_lsp_server
Preserve LSP-specific argv, pinning, environment and permission checks.
src/skillevaluator/plugin_config_risk.py:310
↓ 5 callersFunctionarguments
(tmp_path)
tests/tier3/test_private_comparison_engine.py:15
↓ 5 callersFunctionbuild_tier3_workflow
Clone the expert command with automatic dataset preparation enabled. Keeping its parameters and callback preserves evaluation options, result
src/skillevaluator/tier3/workflow.py:370
↓ 5 callersFunctionbundle
(*, static=False)
tests/reporting/test_private_runtime_render.py:88
↓ 5 callersFunctioncheck_tool_efficiency
Measure what fraction of scored tool calls were productive (planning calls are not scored).
src/skillevaluator/tier3/eval_core/checks.py:8353
↓ 5 callersFunctionclassify_ref
Classify one ``skills``/``rules`` reference (``kind``) without network access. ``listings`` caches directory listings across the refs of one mani
src/skillevaluator/plugin_dependencies.py:605
↓ 5 callersMethodclose
(self)
src/skillevaluator/reporting/console_ui.py:236
↓ 5 callersFunctioncodex_accepts_path
Whether Codex keeps one manifest path value (``resolve_manifest_path`` in the Codex loader). The path must start with ``./``, must not be ``./``,
src/skillevaluator/plugin_formats.py:283
↓ 5 callersMethodcollect
(self)
src/skillevaluator/plugin_discovery.py:228
↓ 5 callersFunctioncollect_mcp_declarations
Collect every MCP server a plugin declares, in Claude Code load order. Claude Code loads ``.mcp.json`` at the plugin root first, then each declar
src/skillevaluator/plugin_mcp.py:201
↓ 5 callersFunctioncombine_behavior_check
( judge_result: dict[str, Any], declarative_results: list[dict[str, Any]], judged_count: int, )
src/skillevaluator/tier3/eval_core/declarative_checks.py:298
↓ 5 callersFunctioncompare_versions
Compute paired version deltas without changing either retained run.
src/skillevaluator/tier3/version_comparison.py:138
↓ 5 callersFunctioncomplete_report
Compact native shape, containing no task prompts or model output.
tests/ci/test_tier3_execution_complete.py:12
↓ 5 callersMethodcomponent_for
(self, file_path: object, metadata: object = None)
src/skillevaluator/reporting/plugin_static.py:88
↓ 5 callersMethodcomponent_inventory
(self)
src/skillevaluator/plugin_inventory.py:135
↓ 5 callersMethodcontent_hashes_for_arm
Stage ``arm`` (``none``/``skills``/``loose``/``full`` or the arm key) privately and hash its pieces.
src/skillevaluator/tier3/loose_parts.py:389
↓ 5 callersFunctioncost_chars
Token-weighted length: 1 per character, 4 per CJK or other full-width character (about 1 token each). Characters divided by 4 fits English text.
src/skillevaluator/plugin_context_cost.py:174
↓ 5 callersFunctioncustom_backend_names
Return the ``--env-mode`` names contributed by installed custom execution backends.
src/skillevaluator/tier3_environments.py:371
↓ 5 callersFunctiondeclared_value_replaces_default
Whether a declared component field replaces the format's default location. Claude Code merges declared fields with its defaults. Cursor replaces
src/skillevaluator/plugin_formats.py:301
↓ 5 callersFunctiondecode_content
(value: dict)
src/skillevaluator/ci_onboarding.py:466
↓ 5 callersFunctiondiscover_and_extract_detailed
Extract entries while accounting for every selected manifest. Unsafe paths, read failures and resource limits continue to abort discovery; on
src/skillevaluator/embedding/extractor.py:264
↓ 5 callersFunctiondiscover_openclaw_skill_manifests
Discover only declared OpenClaw roots, stopping recursively at skill boundaries. Dot-prefixed child directories and ``node_modules`` are not desc
src/skillevaluator/plugin_skills.py:27
↓ 5 callersMethoddownload_artifact
(self, job_id)
ci/feedback.py:672
↓ 5 callersFunctiondrop_token_usage_cases
Empty the per-case rows of ``usage`` in place and return how many were dropped. The report payload drops them first when it would pass its limits
src/skillevaluator/tier3/token_usage.py:656
↓ 5 callersFunctione
()
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 5 callersMethodensure_system_dependencies
( self, environment: BaseEnvironment, dependencies: tuple[str, ...], )
src/skillevaluator/tier3/harbor/installed_agents.py:147
↓ 5 callersFunctionenv_defaults_text
The text *value* carries when every referenced variable is unset: defaults stay, references go.
src/skillevaluator/validators/url_policy.py:246
↓ 5 callersFunctionevidence_ref_identity
Return the most specific stable location carried by an evidence ref.
src/skillevaluator/evidence.py:12
↓ 5 callersMethodexec_as_agent
( self, environment: BaseEnvironment, command: str, env: dict[str, str] | None
src/skillevaluator/tier3/harbor/local_agents.py:510
↓ 5 callersMethodextract_json_from_response
Send a completion and parse one unambiguous JSON object from the response. The whole reply, optionally in a Markdown fence, is parsed first.
src/skillevaluator/inference/client.py:852
↓ 5 callersFunctionfailure_reason_text_slots
Yield each ``(container, key)`` that holds a built failure reason or excerpt. Order follows the report: evaluator-card evidence per agent, then t
src/skillevaluator/evaluation/tier3_findings.py:302
↓ 5 callersFunctionfake_api
(monkeypatch, gate, payload)
tests/ci/test_gitlab_default_head_admission.py:38
← previousnext →1,801–1,900 of 24,741, ranked by callers