MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 3 callersFunctionlegacy_results_root
Return the historical in-skill results directory.
src/skillevaluator/tier3/results_location.py:54
↓ 3 callersFunctionlift_row
The skill-lift row with paired horizontal score bars.
src/skillevaluator/reporting/console_ui.py:278
↓ 3 callersMethodlisten
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 3 callersMethodlisting
(self, label: str, raw: object)
src/skillevaluator/reporting/native_evidence.py:152
↓ 3 callersMethodliteral
Decode a JSON-compatible literal: no templates, spreads, computed keys or comments.
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:961
↓ 3 callersFunctionload
(name)
tests/ci/test_github_api.py:16
↓ 3 callersMethodload
Load ``path``; a file that does not load is kept as an error and applies no waivers.
src/skillevaluator/validators/pii_waivers.py:267
↓ 3 callersFunctionload_agent_eval_payload
Build the canonical agent-eval payload for one run without new judge calls. ``use_llm_judge=False`` keeps the report builder from calling the Ins
src/skillevaluator/evaluation/tier3_feedback_chat.py:205
↓ 3 callersFunctionload_condition_rows
Restore collector-owned annotations from their public persistence envelope.
src/skillevaluator/tier3/baseline_store.py:207
↓ 3 callersFunctionlocal_subprocess_env
Build a host env for Harbor local subprocesses with runtime bins first.
src/skillevaluator/tier3/harbor/local_runtime.py:184
↓ 3 callersFunctionlooks_like_random_secret
A long random-looking value (``SERVICE_SEED=9f2c...``) that may be key material whatever its key says. 32 or more hex or base64 characters with h
src/skillevaluator/validators/url_policy.py:340
↓ 3 callersFunctionlooks_like_secret_material
Whether a value reads as key material by its shape, whatever its key says. Three character classes out of lower case, upper case, digits, and oth
src/skillevaluator/validators/url_policy.py:324
↓ 3 callersFunctionmain
Classify one pull-request diff and emit a GitHub Actions output.
scripts/classify_ci_changes.py:181
↓ 3 callersFunctionmain
Print the selected test files, one per line.
scripts/select_docs_tests.py:110
↓ 3 callersFunctionmanaged_gitleaks_path
Where the managed Gitleaks binary lives; None when the runtime root is invalid.
src/skillevaluator/utils/scanner_install.py:151
↓ 3 callersFunctionmanifest_identity
The stable plugin identity a manifest declares: OpenClaw's ``id``, otherwise ``name``. OpenClaw's ``name`` is an optional display label, never an
src/skillevaluator/plugin_formats.py:267
↓ 3 callersFunctionmanifest_root_for
Return the plugin root that a manifest path names, or ``None`` if *path* names no manifest. A lexical check (see :func:`manifest_relative_path`):
src/skillevaluator/plugin_manifest.py:398
↓ 3 callersFunctionmanifest_type_for_relative_path
Return the manifest type of one root-relative manifest path (case variants of client manifests included).
src/skillevaluator/plugin_manifest.py:412
↓ 3 callersFunctionmatches
(name: str)
src/skillevaluator/tier3/eval_core/plugin_signals.py:3633
↓ 3 callersFunctionmcp_message_prefix
The ``mcpServers['<name>']: `` prefix of a named server's finding messages (name redacted and bounded). Callers that reword these findings (LSP s
src/skillevaluator/validators/mcp_static.py:389
↓ 3 callersFunctionmcp_server_is_read_only
Whether a runnable MCP server declares a read-only mode (flag or env). The flag counts when it is bare or set to a true value; ``--read-only=fals
src/skillevaluator/plugin_component_risk.py:704
↓ 3 callersFunctionme
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 3 callersMethodmetadata
(self, handle: int)
tests/utils/test_secure_fs_platform.py:345
↓ 3 callersFunctionmock_tool_unavailable
Context manager to mock a tool as unavailable.
tests/validators/test_security_validators.py:27
↓ 3 callersFunctionnegated_before
True when advice that begins at ``start`` is negated ("do not ...", "without first ...", "instead of ...").
src/skillevaluator/evaluation/dataset_case_removal.py:87
↓ 3 callersFunctionnemo_gym_harbor_only_option_errors
Reject baseline reuse, version comparison, budgets and baseline-timeout tolerance until Gym can honor them.
src/skillevaluator/tier3/nemo_gym/backend.py:238
↓ 3 callersFunctionnemo_gym_settings_errors
Reject invalid ``SKILL_EVAL_NEMO_GYM_*`` settings, before any dataset work.
src/skillevaluator/tier3/nemo_gym/backend.py:273
↓ 3 callersFunctionno_fallback_stream
The same refusal with CLAUDE_CODE_DISABLE_REFUSAL_FALLBACK=1.
tests/claude_refusal_fixtures.py:146
↓ 3 callersFunctionnormalize_tool_call
Unwrap proven native Codex calls without interpreting arbitrary JavaScript.
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:818
↓ 3 callersMethodof
(cls, inventory: PluginInventory)
src/skillevaluator/tier3/plugin_prepare.py:74
↓ 3 callersFunctionopenclaw_ssl_verification_enabled
Effective OpenClaw TLS policy: the canonical boolean, then the legacy one, else verified TLS.
src/skillevaluator/validators/mcp_static.py:1265
↓ 3 callersMethodoverride
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 3 callersFunctionp
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 3 callersFunctionparse
()
tests/utils/test_structured_data.py:63
↓ 3 callersFunctionparse_export
Parse a v1 wrapper or JSONL; malformed lines become unsupported records.
src/skillevaluator/tier3/trace_import.py:153
↓ 3 callersFunctionparse_mcp_runner
The package runner an MCP declaration runs, or ``None`` when it runs none. This is the one argv reader for package runners: the pinning check
src/skillevaluator/validators/mcp_static.py:2084
↓ 3 callersFunctionparse_name_status_z
Parse ``git diff --name-status -z`` without losing rename sources.
scripts/classify_ci_changes.py:95
↓ 3 callersMethodpathSegment
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 3 callersFunctionpaths_refer_to_same_location
Compare output paths after resolving aliases, symlinks, and hard links.
src/skillevaluator/utils/tier2_paths.py:105
↓ 3 callersFunctionpayload
(index: str, element)
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:1027
↓ 3 callersFunctionplain_check_name
Return a readable name for a check that reports a raw ID such as ``SCRIPT_LINT``.
src/skillevaluator/reporting/html.py:1987
↓ 3 callersFunctionplan
(gateway)
tests/test_gateway_defaults.py:113
↓ 3 callersMethodpost
(self, path, payload)
ci/feedback.py:625
↓ 3 callersFunctionprint_history
(history: dict, console: Console)
src/skillevaluator/tier3/history.py:437
↓ 3 callersFunctionprint_link
(url: str)
src/skillevaluator/ci_onboarding.py:60
↓ 3 callersMethodprocess
Orchestrate a full LLM interaction: prompt -> call -> parse. On failure the fallback response is returned so callers always receive a
src/skillevaluator/inference/client.py:909
↓ 3 callersFunctionprompt
(content: str)
tests/tier3/test_judge_contracts.py:1574
↓ 3 callersFunctionpublication_evidence_dict
Project an untrusted marker to the four canonical fields safe for reports.
src/skillevaluator/publication_evidence.py:86
↓ 3 callersFunctionpublication_target_for_results
Return the one canonical target shared by publication evidence.
src/skillevaluator/reporting/base.py:967
↓ 3 callersMethodput
(self, path, payload)
tests/ci/test_feedback.py:665
↓ 3 callersMethodread
(self, _limit: int)
tests/test_harbor_secure_docker_environment.py:177
↓ 3 callersFunctionread_nvidia_build_key_from_stdin
Read and cache the parent-provided key without storing it on disk.
src/skillevaluator/tier3/harbor/sensitive_stdin.py:26
↓ 3 callersFunctionread_results
Validate exact retained native JSON; never extract or interpret HTML.
ci/feedback.py:122
↓ 3 callersFunctionread_verifier_rewards
Return a trial's rewards from ``reward.json``, else a numeric ``reward.txt``, as Harbor reads them.
src/skillevaluator/tier3/nemo_gym/results.py:90
↓ 3 callersFunctionrecord
(bucket: str, relative: Path, reason: str)
ci/diagnostics.py:235
↓ 3 callersFunctionrecord_similarity
(result, key, *, status, catalog_entries, matches, reason=None)
src/skillevaluator/deduplication/plugin/catalog_checks.py:32
↓ 3 callersFunctionrecorded_not_applicable_reason
Return the reason *reward* records for its N/A *metric*, or ``None`` when it records none.
src/skillevaluator/tier3/harbor/metrics.py:397
↓ 3 callersFunctionrecovery_lines
Return the terminal lines that report recovered agent timeouts.
tests/test_harbor_timeout_recovery.py:389
↓ 3 callersMethodrefresh
(self)
tests/test_tier3_progress.py:1915
↓ 3 callersFunctionrefresh_host_vertex_adc_environment
Refresh OPENAI_API_KEY in-place when provenance is ADC and endpoint is Vertex OpenAPI.
src/skillevaluator/provider_config.py:444
↓ 3 callersMethodregister
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 3 callersMethodrender
Render single result to string (captures console output).
src/skillevaluator/reporting/cli.py:163
↓ 3 callersMethodrender
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 3 callersMethodrender_all
Render all results to JSON with overall summary.
src/skillevaluator/reporting/json_reporter.py:142
↓ 3 callersFunctionrender_html
(root: Path)
tests/test_harbor_timeout_recovery.py:394
↓ 3 callersFunctionrender_native_retained_report
Render every tier from typed JSON, then constrain trusted generated code. Tier 3 templates contain JSON-escaped data in dynamic scripts and handl
ci/native_reports.py:116
↓ 3 callersFunctionrender_token_usage_table
Print one row per agent and the all-agent row, then the notes; return whether anything printed. ``case_pointer`` names the files of the calling c
src/skillevaluator/reporting/tier3_token_usage.py:305
↓ 3 callersFunctionreport_path
A SkillSpector report path as POSIX text relative to the scan, else ``None``. The path is checked lexically and never resolved, because the file
src/skillevaluator/skillspector_references.py:158
↓ 3 callersFunctionrequire
(condition: bool, message: str)
scripts/demo_starter.py:32
↓ 3 callersMethodresize
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 3 callersFunctionresolve_acceptable_skills
Resolve expected plus acceptable alternate skill names from a dataset entry.
src/skillevaluator/tier3/eval_core/checks.py:3227
↓ 3 callersFunctionresolve_tool_path
Resolve a tool binary from PATH and the interpreter's own bin dir. Isolated installs (``uv tool install``, ``pipx``) place the console script
src/skillevaluator/utils/tool_runner.py:63
↓ 3 callersFunctionresult_event
(text: str, *, status: int | None = None)
tests/claude_refusal_fixtures.py:76
↓ 3 callersFunctionresult_tier
(result: ValidationResult)
src/skillevaluator/reporting/run_status.py:130
↓ 3 callersFunctionreused_baseline_note
Say that the baseline tokens come from an earlier run when this run reused its baseline.
src/skillevaluator/reporting/tier3_token_usage.py:149
↓ 3 callersFunctionrewrite
(value)
src/skillevaluator/tier1/plugin_checks.py:84
↓ 3 callersMethodrotate
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 3 callersFunctionroute_evidence
(config, env, workspace)
tests/ci/test_gitlab_enriched_admission.py:105
↓ 3 callersFunctionrun_cancelled
()
tests/test_harbor_secure_docker_environment.py:7319
↓ 3 callersFunctionrun_cleanup
()
tests/test_harbor_local_mode.py:3606
↓ 3 callersFunctionrun_dependency_audits
Explicitly audit normalized inputs, preserving incomplete source coverage.
src/skillevaluator/plugin_dependency_audit.py:609
↓ 3 callersFunctionrun_quality_check
( target_path: Path, *, min_score: int = QUALITY_DEFAULT_MIN_SCORE, quality_config: Path | None = None )
src/skillevaluator/tier1/commands.py:554
↓ 3 callersFunctionrun_record
Return the run-level record: the tolerated agents and a bounded sample of the launch errors they explain. ``explained_launch_errors_total`` is th
src/skillevaluator/tier3/harbor/baseline_timeouts.py:272
↓ 3 callersMethodruns_past
Return whether the assignment rule masks past *end* the value of a credential name in *start*:*end*, or at *end*. Masking the value *start*:*
src/skillevaluator/tier3/eval_core/checks.py:4785
↓ 3 callersFunctionruntime_path
Return PATH with any bring-your-own runtime bins prepended.
src/skillevaluator/tier3/harbor/local_runtime.py:139
↓ 3 callersFunctionsafe_progress_reporter
Return an idempotent best-effort wrapper around a progress reporter.
src/skillevaluator/tier3/harbor/progress.py:1025
↓ 3 callersFunctionsafe_relative
(value: Any)
src/skillevaluator/tier3/trace_import.py:181
↓ 3 callersFunctionscan
(file_path: str, line: int)
tests/reporting/test_reporters.py:860
↓ 3 callersFunctionscan_pids
(output, deadline)
tests/ci/test_run_tier1.py:1503
↓ 3 callersMethodscoped_exec_env
(self, values: dict[str, str])
tests/test_harbor_local_mode.py:1575
↓ 3 callersFunctionscore_skill_execution
Compute execution credit, requiring activation for positive cases. Activated or unresolved cases retain the average of sub-checks. A definite
src/skillevaluator/tier3/eval_core/checks.py:8457
↓ 3 callersFunctionscored_goal_accuracy
Return a scored goal-accuracy value, excluding explicit N/A and errors.
src/skillevaluator/tier3/attempt_verdict.py:33
↓ 3 callersFunctionselect_tolerated
Decide the run-level outcome (T8); returns ``(tolerated, rejected, explained_launch_errors)``. ``agents`` is the collected agent payloads, ``cand
src/skillevaluator/tier3/harbor/baseline_timeouts.py:204
↓ 3 callersMethodset_cdata_mode
(self, elem: str, *, escapable: bool = False)
src/skillevaluator/validators/markdown.py:164
↓ 3 callersFunctionshell_path
Quote a census target for sh: a leading ``$VAR`` expands, the rest is literal.
src/skillevaluator/tier3/plugin_native.py:877
↓ 3 callersFunctionskill_root_changed
Return whether any changed path (both sides of a rename) is inside a skill root. The paid jobs (LLM-enriched Tier 1, Tier 2 and Tier 3) run on a
ci/select_changed_skills.py:45
↓ 3 callersFunctionskipped_result
(name, description, key, reason, *, catalog_entries=0)
src/skillevaluator/deduplication/plugin/catalog_checks.py:45
↓ 3 callersFunctionskipped_summary
()
tests/ci/test_publish_reports.py:1225
↓ 3 callersFunctionsource_validation_preview
Return the failed Tier 3 source checks a terminal view prints, and the note after them. A terminal view previews the first ``SOURCE_VALIDATION_CL
src/skillevaluator/reporting/base.py:2630
↓ 3 callersFunctionsplit_npm_spec
``(name, version)`` of an npm registry spec; ``version`` is ``None`` when the spec names none. A scope's leading ``@`` belongs to the name: ``@sc
src/skillevaluator/validators/mcp_static.py:2252
← previousnext →3,301–3,400 of 24,741, ranked by callers