MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 2 callersMethodallows_host
Whether an entry covers ``host``, whose static class is ``endpoint`` (``None`` when it looks public). A public-looking host matches by name o
src/skillevaluator/validators/mcp_static.py:3024
↓ 2 callersFunctionalternatives
(depth)
src/skillevaluator/tier3/eval_core/plugin_security.py:2763
↓ 2 callersFunctionanalyze_agent
Record one subagent's grants and flag risky ones (the agent is never loaded). ``plugin_name`` builds the ``mcp__plugin_<plugin>_<server>`` names
src/skillevaluator/plugin_component_risk.py:835
↓ 2 callersFunctionanalyze_command
Record one command's pre-approved tools and flag unrestricted Bash or wildcard grants. ``entry`` is a ``plugin.json`` ``commands`` map entry, who
src/skillevaluator/plugin_component_risk.py:962
↓ 2 callersFunctionanalyze_monitor
Analyze an unsandboxed monitor command and its permissions, without starting it.
src/skillevaluator/plugin_config_risk.py:391
↓ 2 callersFunctionanalyze_skill
Record one skill's pre-approved tools (``allowed-tools``) and flag unrestricted Bash or wildcard grants. Claude Code pre-approves a plugin skill'
src/skillevaluator/plugin_component_risk.py:984
↓ 2 callersFunctionannounce
(message: str)
src/skillevaluator/tier3/dataset_progress.py:29
↓ 2 callersFunctionao
(t,e,i,s)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersMethodapplyStack
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersMethodarm
(self, hook: Any)
tests/test_tier3_progress.py:2430
↓ 2 callersMethodarm_enabled
(self, arm: str)
src/skillevaluator/tier3/eval_core/plugin_signals.py:1332
↓ 2 callersFunctionassert_completed_run_dir
Return the resolved run directory, or raise when it is not a completed Tier 3 run.
src/skillevaluator/evaluation/tier3_feedback_chat.py:182
↓ 2 callersFunctionattach_metric_evidence_refs
Attach evidence refs to existing metric detail dictionaries in place.
src/skillevaluator/tier3/eval_core/atif_helpers.py:879
↓ 2 callersFunctionattempt_denominators
(group: dict[str, Any])
src/skillevaluator/reporting/declarative_coverage.py:57
↓ 2 callersFunctionbalanced_json_container_end
Return the exclusive end of the string-aware ``{...}``/``[...]`` span at *start*. ``None`` means *start* is not a container opener, or the span n
src/skillevaluator/inference/json_span.py:18
↓ 2 callersMethodbeforeLayout
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersFunctionbehavior_evidence_gap
Describe the required behavior evidence missing from a bundle, or ``""``. A behavior score read off evidence whose final answer was silently
src/skillevaluator/tier3/eval_core/atif_helpers.py:1258
↓ 2 callersFunctionbenchmark_documents
Compose trusted native JSON in memory; never trust artifact Markdown. The caller validates the CI manifest and bounds the native JSON mapping.
ci/benchmark_reports.py:146
↓ 2 callersMethodbindEvents
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersMethodbind_execution
( self, case_ids: Sequence[str], *, n_attempts: int, stop_on_pass: bool, pass_threshold: float )
src/skillevaluator/tier3/plugin_comparisons.py:274
↓ 2 callersFunctionblob_id
(content: str)
src/skillevaluator/ci_onboarding.py:147
↓ 2 callersFunctionbrand_html
Logo and product name for the rail of the CI index and the Pages history. The logo is decorative next to the visible name; ``uid`` is kept so a
ci/native_reports.py:179
↓ 2 callersFunctionbuild
(*args: Any, **kwargs: Any)
tests/tier3/test_view_results_feedback.py:305
↓ 2 callersMethodbuildLabels
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersFunctionbuild_agent_statistics
Assemble every C4 statistics block for one agent. ``arms`` is keyed by ``with_skill``, ``without_skill``, ``sum_of_parts`` and ``loose_parts`
src/skillevaluator/tier3/harbor/stats.py:1565
↓ 2 callersFunctionbuild_dataset_diagnostics_view
Add exact explanation groups inside each retained attempt and status. No records are scored, combined across attempts, or newly bounded here.
src/skillevaluator/reporting/diagnostic_view.py:53
↓ 2 callersFunctionbuild_insights_view
Project concise failed-run cards, retaining original diagnostics separately. Healthy feedback is copied verbatim. Failed runs replace only recogn
src/skillevaluator/reporting/insights_view.py:94
↓ 2 callersFunctioncalculate_full_jitter_delay
Calculate delay using exponential backoff with full jitter (AWS standard).
src/skillevaluator/inference/retry.py:74
↓ 2 callersFunctioncall
(prompt, **kwargs)
tests/test_tier3_case_summary.py:1966
↓ 2 callersFunctioncall
(prompt: str, **kwargs: Any)
tests/tier3/test_judge_contracts.py:61
↓ 2 callersFunctioncall_model
(candidate_model, provenance)
src/skillevaluator/tier3/harbor/templates/eval.py:2438
↓ 2 callersFunctioncall_public_llm
Call the configured public provider through the shared client. The shared client is responsible for OpenAI-compatible endpoints, Anthropic, a
src/skillevaluator/tier3/eval_core/llm_judge.py:256
↓ 2 callersFunctioncanary_leak_rate
Return the exact unrounded rate, or ``None`` for unknown outcomes/counts. A legacy summary without ``planted`` uses ``n_trials`` as its denominat
src/skillevaluator/tier3/eval_core/runtime_evidence.py:415
↓ 2 callersFunctioncanonical_dimension_mean
Average one complete canonical dimension set, failing closed on gaps.
src/skillevaluator/tier3/harbor/metrics.py:550
↓ 2 callersFunctioncard
()
tests/ci/test_benchmark_report_links.py:25
↓ 2 callersMethodcase_spec
(self, case_id: str)
src/skillevaluator/tier3/eval_core/plugin_signals.py:1407
↓ 2 callersFunctioncatalog_payload
(client, *, version=2, skills=(), plugins=())
tests/deduplication/plugin/test_private_dedup.py:98
↓ 2 callersFunctioncell
(value: object)
src/skillevaluator/reporting/plugin_static_render.py:38
↓ 2 callersFunctionchanged_paths
Return every path changed from the merge base through ``head``.
scripts/classify_ci_changes.py:138
↓ 2 callersFunctionchangelog_entry
Return the release heading, subsection heading and plain text of the one entry starting with ``lead``.
tests/changelog_entries.py:19
↓ 2 callersFunctioncharacter_facts_enabled
Read the operator's opt-in; existing prompts remain unchanged by default.
src/skillevaluator/tier3/eval_core/character_facts.py:12
↓ 2 callersMethodchartOptionScopes
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersFunctioncheck_files
(repo: Path, files: dict[str, str], expected: dict[str, str] | None = None)
src/skillevaluator/ci_onboarding.py:280
↓ 2 callersMethodcheck_listing
Stop listing ``directory`` once its names must exceed the path budget. An excluded directory is the only entry that does not consume the
src/skillevaluator/utils/secure_fs.py:1001
↓ 2 callersFunctioncheck_script_execution
Check whether the agent executed the expected script.
src/skillevaluator/tier3/eval_core/checks.py:7152
↓ 2 callersFunctioncheck_status
(item: dict[str, Any])
src/skillevaluator/reporting/declarative_coverage.py:53
↓ 2 callersFunctioncheck_ticker_row
Live Tier 1 lineup: completed checks tick, the active one is marked. The tier header's spinner carries the animation; the active check gets a
src/skillevaluator/reporting/console_ui.py:883
↓ 2 callersFunctioncheckout
(job: str)
tests/ci/test_publisher_workflow.py:364
↓ 2 callersFunctionci
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersFunctionclassify
(reward, metric, thresholds)
src/skillevaluator/tier3/grader_validation.py:298
↓ 2 callersMethodclearCache
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersFunctionclock
(monkeypatch: pytest.MonkeyPatch)
tests/tier3/test_installed_agent_install_retry.py:122
↓ 2 callersMethodclose
(self)
tests/test_harbor_local_mode.py:805
↓ 2 callersFunctioncn
(t,...e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersFunctioncollect
(key: str, allowed: frozenset[str], rank: dict[str, int])
src/skillevaluator/reporting/base.py:665
↓ 2 callersFunctioncollect
(source: SecureRoot, relative: Path, target: Path, metadata: os.stat_result)
ci/diagnostics.py:241
↓ 2 callersFunctioncollect_native_evidence
Per-agent load evidence of the native with arms, read from the retained Harbor jobs. A trial without a census file contributes only its staging p
src/skillevaluator/tier3/plugin_execution.py:536
↓ 2 callersFunctioncollect_sensitive_bytes
Use the same bounded credential preflight for files and trusted Git blobs.
ci/secret_redaction.py:61
↓ 2 callersFunctioncombine_reference_summaries
The totals of the valid *summaries*, such as those of each skill in a folder.
src/skillevaluator/skillspector_references.py:463
↓ 2 callersMethodcommand_before
Whether ``command_re`` matches before ``start`` in the shell command that holds ``start``.
src/skillevaluator/validators/mcp_static.py:3237
↓ 2 callersFunctioncommitted_blob
Hash using the same path filters Git applies when staging this file.
src/skillevaluator/ci_onboarding.py:156
↓ 2 callersFunctioncompact_archive
(platform)
tests/ci/test_feedback.py:235
↓ 2 callersFunctioncomparison_mismatches
Explain all baseline-relevant differences, ignoring skill revisions.
src/skillevaluator/tier3/baseline_store.py:168
↓ 2 callersFunctioncompletion_token_limit
Return ``max_tokens`` raised to the reasoning ceiling for OpenAI reasoning models.
src/skillevaluator/provider_config.py:186
↓ 2 callersFunctioncompute_dimensions
Compute the 5 human-readable dimensions from raw evaluator data. Args: evaluators: Raw evaluator scores from SkillEvaluator. case
src/skillevaluator/evaluation/dimension_judge.py:413
↓ 2 callersMethodconfigure_run
( self, *, case_id_by_task_selector: Mapping[str, str] | None = None, arm_jobs
src/skillevaluator/tier3/plugin_evidence.py:334
↓ 2 callersMethodconfigure_run
Supply coordinator-owned identities and additional private arm locations.
src/skillevaluator/tier3/plugin_execution.py:92
↓ 2 callersFunctionconfigured_pii_categories
Return the PII check names in the PII pattern config; the loader also refuses credential checks.
src/skillevaluator/validators/pii_waivers.py:89
↓ 2 callersMethodconnect
(self)
src/skillevaluator/model_catalog.py:226
↓ 2 callersMethodconnect
(self)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:613
↓ 2 callersFunctioncontained
(root: Path, relative: str)
src/skillevaluator/tier3/trace_import.py:190
↓ 2 callersFunctioncontainer_id
(service: str)
tests/test_harbor_secure_docker_environment.py:8063
↓ 2 callersFunctioncontext
(env: dict)
ci/run_tiers.py:131
↓ 2 callersFunctioncounter
(value: object)
src/skillevaluator/reporting/html.py:1350
↓ 2 callersMethodcover
Record that the PII scan covered ``unit``: one skill, a folder scanned directly, or a file. Only waivers whose file lies in a covered unit ar
src/skillevaluator/validators/pii_waivers.py:289
↓ 2 callersFunctioncreate_progress_reporter
Create the requested presentation; ``auto`` follows stream TTY state. ``refresh_interval`` overrides the heartbeat period, which otherwise de
src/skillevaluator/tier3/harbor/progress.py:1032
↓ 2 callersFunctioncredential_probe_disposition
Classify a live catalog probe without rejecting compatible custom gateways.
src/skillevaluator/tier3/harbor/runtime_preflight.py:776
↓ 2 callersFunctionct
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 2 callersFunctioncurrent
()
ci/benchmark_publish.py:135
↓ 2 callersFunctiondataset_generation_progress
Report elapsed waiting time; never imply provider-side progress or a deadline.
src/skillevaluator/tier3/dataset_progress.py:19
↓ 2 callersFunctiondataset_snapshot_manifest
Return validated dataset metadata suitable for embedding in ``result.json``.
src/skillevaluator/tier3/harbor/report_data.py:191
↓ 2 callersMethoddeclared
(self)
src/skillevaluator/tier3/plugin_native.py:832
↓ 2 callersFunctiondeclared_mcp_server_matcher
:func:`match_declared_mcp_server` for one set of declared servers, indexed once for many lookups.
src/skillevaluator/tier3/eval_core/plugin_signals.py:2245
↓ 2 callersFunctiondeclared_plugin_paths
The root-relative paths a ``plugin.json`` path field declares (one path or a list of them). Each path is normalized with the Tier 1 inventory's r
src/skillevaluator/tier3/plugin_native.py:511
↓ 2 callersFunctiondeclares_agent_plugins_schema
Whether a parsed root ``plugin.json`` opts into Agent Plugins semantics. Clients such as VS Code read a root ``plugin.json`` with Agent Plugins
src/skillevaluator/plugin_formats.py:49
↓ 2 callersFunctiondecode_manifest_leniently
Decode manifest bytes the way a lenient client does, never failing. A UTF-32 or UTF-16 byte-order mark, or the NUL pattern of BOM-less UTF-16
src/skillevaluator/plugin_manifest.py:157
↓ 2 callersFunctiondedupe_rows
Drop rows reloaded from the same resolved source; keys are unique within one parse.
src/skillevaluator/reporting/dashboard_data.py:461
↓ 2 callersFunctiondependency_records
Classify all original manifest refs locally; only proven missing targets block.
src/skillevaluator/plugin_dependency_records.py:133
↓ 2 callersFunctiondescribe_issue_counts
Describe a report's issue counts in the shared HTML and Markdown wording. Unique findings and their distinct occurrences come first; ``raw_record
src/skillevaluator/reporting/finding_summary.py:325
↓ 2 callersMethoddiff
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersFunctiondimension_is_not_applicable
Return whether every source metric of *dimension* is N/A.
src/skillevaluator/tier3/harbor/metrics.py:437
↓ 2 callersMethoddiscover_latest_report
Return the latest HTML report path for a skill, or None if absent. Looks for ``report.html`` in the latest results directory, then falls
src/skillevaluator/evaluation/service.py:107
↓ 2 callersFunctiondisplay_findings_report
Display the findings report panel after the score table. Return the atomic feedback messages printed in the detailed panel. The caller uses t
src/skillevaluator/tier3/harbor/report.py:1288
↓ 2 callersMethoddownload_dir_filtered
( self, *, source_dir: str, target_dir: Path | str, include: Sequence[
src/skillevaluator/tier3/harbor/local_environment.py:890
↓ 2 callersMethoddownload_dir_with_exclusions
( self, *, source_dir: str, target_dir: Path | str, exclude: list[str]
src/skillevaluator/tier3/harbor/local_environment.py:864
↓ 2 callersMethoddownload_file
(self, source_path: str, target_path: Path | str)
tests/test_harbor_claude_refusal_fallback.py:580
↓ 2 callersMethoddrain
(self)
tests/test_secure_docker_environment.py:132
↓ 2 callersMethoddrawBackground
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersMethoddrawBorder
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 2 callersMethoddrawGrid
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
← previousnext →4,901–5,000 of 24,741, ranked by callers