MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 9 callersFunction_security_card
(payload: dict[str, Any])
tests/reporting/test_tier3_failure_reasons.py:122
↓ 9 callersFunction_semantic_text
Normalize compatibility characters and remove invisible spoofing characters.
scripts/ci/check_public_benchmarks.py:1807
↓ 9 callersFunction_shell_tokens
(cmd: Any)
src/skillevaluator/tier3/eval_core/shell_lexing.py:193
↓ 9 callersFunction_stage
(tmp_path, source, **kwargs)
tests/tier3/test_native_mcp_policy.py:32
↓ 9 callersFunction_status
(skill: Path)
tests/harness/test_overlay.py:39
↓ 9 callersFunction_step
(tool_calls: list[dict[str, Any]], results: list[dict[str, Any]] | None = None)
tests/tier3/test_plugin_signals.py:55
↓ 9 callersFunction_steps_chart
(html: str)
tests/reporting/test_tier3_baseline_trial_views.py:191
↓ 9 callersFunction_text
(value: object)
ci/report_views.py:33
↓ 9 callersFunction_tier3_payload
(output: str)
tests/test_harbor_html_report_failures.py:40
↓ 9 callersFunction_validate
(skill: Path, out: Path, *args: str)
tests/golden/test_policy_behavior.py:43
↓ 9 callersFunction_validate_document
Scan a supporting document through the public Tier 1 orchestration.
tests/validators/test_hygiene_link_boundaries.py:22
↓ 9 callersMethod_validate_single_skill
Run all schema validation checks on a single skill directory.
src/skillevaluator/validators/schema.py:364
↓ 9 callersFunction_verify_entry
(entry: _ManifestEntry, metadata: os.stat_result, path: Path)
src/skillevaluator/tier3/harbor/secure_copy.py:691
↓ 9 callersFunction_wilson_score_interval
Return a two-sided 95% Wilson score interval for a binomial rate.
src/skillevaluator/tier3/harbor/collector.py:4834
↓ 9 callersFunction_write_job_result
(jobs_dir: Path, stats: dict[str, object], *, total: int = 1)
tests/test_harbor_runner_status.py:2340
↓ 9 callersFunction_write_skill
(tmp_path: Path, scripts: dict[str, str], skill_md_body: str = "# Skill\n")
tests/tier3/test_dataset_primary_script.py:23
↓ 9 callersFunctionarms
(cases=("a", "b", "c", "d", "e"))
tests/tier3/test_private_comparison_contracts.py:23
↓ 9 callersFunctionaverage_metrics
Average SkillEvaluator metrics across rewards, with legacy artifact compatibility.
src/skillevaluator/tier3/harbor/metrics.py:512
↓ 9 callersFunctionbuild_case_diagnostics
( dataset: list[dict[str, Any]], agents: dict[str, dict[str, Any]], *, evaluation_mode: str = "both" )
src/skillevaluator/evaluation/tier3_expectations.py:173
↓ 9 callersMethodbuild_from_paths
Replace the index with explicit skills under one trusted checkout. Paths retain their checkout-relative identity, avoiding collisions when
src/skillevaluator/embedding/registry.py:256
↓ 9 callersFunctionchat_completion_to_responses_events
Yield terminal Responses API events for one non-streaming Chat Completion.
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1201
↓ 9 callersMethodclose
(self)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:108
↓ 9 callersFunctioncollect_sensitive_values
Find credential literals using the same patterns as the PII scanner. We scan matches even in comments or configured exceptions: another scanner
ci/secret_redaction.py:42
↓ 9 callersFunctioncompact_header
(markdown)
tests/ci/test_report_views.py:83
↓ 9 callersMethodcosine_similarity
Compute cosine similarity between two vectors. Uses stdlib math only -- no numpy dependency required. Returns: Similarit
src/skillevaluator/embedding/client.py:288
↓ 9 callersMethoddeduct
Deduct from score and record the issue.
src/skillevaluator/models/quality.py:50
↓ 9 callersFunctiondefault_policy
Return the default policy for both public and private repositories.
src/skillevaluator/validators/policy.py:324
↓ 9 callersFunctionensure_string_list
Convert various input types to a list of strings. Handles the common pattern of accepting either a single string or a list for fields like ta
src/skillevaluator/models/field_validators.py:13
↓ 9 callersMethodexpect
(self, *tokens: str)
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:932
↓ 9 callersFunctionfailed_result
( *, message="First blocking problem", suggestion="Apply the recorded fix", path="skills/sample/SKILL.md"
tests/ci/test_report_views.py:530
↓ 9 callersFunctionfinalize_publication_target
Stamp results only when the producer's source stayed unchanged.
src/skillevaluator/publication_identity.py:781
↓ 9 callersFunctionfind_evals_file
Return the first supported SkillEvaluator eval dataset for a skill, if present.
src/skillevaluator/tier3/harbor/adapter.py:1893
↓ 9 callersFunctionhelper
()
tests/ci/test_gitlab_feedback_request.py:18
↓ 9 callersFunctionimport_traces
(skill: Path, export: Path, output: Path, *, force: bool = False, dry_run: bool = False)
src/skillevaluator/tier3/trace_import.py:410
↓ 9 callersFunctionincomplete_arms
Return the agent's arms whose recorded ``execution_status`` is ``failed``. A skipped arm (``--skip-baseline`` or ``--skip-with-skill``) did not r
src/skillevaluator/reporting/tier3_coverage.py:97
↓ 9 callersFunctioninventory
(root: Path, manifest: str, data: dict, files: dict)
tests/test_plugin_context_cost.py:13
↓ 9 callersFunctionis_link_or_reparse
Return whether *path* itself is a symlink, junction, or reparse point.
src/skillevaluator/utils/tier2_paths.py:24
↓ 9 callersFunctionload_dataset_entries_with_format
Load normalized dataset entries and report the detected authoring format.
src/skillevaluator/tier3/dataset_utils.py:91
↓ 9 callersFunctionload_policy_file
Load a custom policy YAML, layered on top of *base_profile*. Raises: FileNotFoundError: when *path* does not exist. ValueError: w
src/skillevaluator/validators/policy.py:281
↓ 9 callersFunctionmodule
(name)
tests/ci/test_all_tier_preflight.py:14
↓ 9 callersMethodmonotonic
(self)
tests/test_harbor_runner_status.py:815
↓ 9 callersFunctionparse_agent_model_overrides
Parse ``--agent-model agent=model`` values.
src/skillevaluator/tier3/commands.py:391
↓ 9 callersMethodprocess
(value: str, state: click.parser.ParsingState)
src/skillevaluator/cli.py:471
↓ 9 callersFunctionprofile_for
Return the profile of a contained manifest type (Claude Code by default).
src/skillevaluator/plugin_formats.py:278
↓ 9 callersMethodread_text
(self, *, encoding: str)
tests/reporting/test_unified_tier3_report.py:713
↓ 9 callersFunctionreport
(selected, observed=None, repeats=1)
tests/tier3/test_grader_policy.py:45
↓ 9 callersFunctionreporters
()
tests/reporting/test_private_static_reports.py:32
↓ 9 callersFunctionresolve_git_root
Return the containing Git repository root without importing optional tiers.
src/skillevaluator/utils/helpers.py:146
↓ 9 callersFunctionresolve_should_trigger
Resolve routing while preserving legacy unlabeled cases as ``None``.
src/skillevaluator/tier3/eval_core/checks.py:3235
↓ 9 callersFunctionretry_call_with_backoff
Execute operation, retrying on transient HTTP or network faults using full jitter.
src/skillevaluator/inference/retry.py:207
↓ 9 callersFunctionreviewed
(workspace, decision="accept", **kwargs)
tests/test_trace_import.py:41
↓ 9 callersFunctionruntime_report
()
tests/reporting/test_plugin_evidence_report.py:43
↓ 9 callersFunctionsecurity_check
(request: pytest.FixtureRequest)
tests/tier3/test_agent_output_cleanup_scope.py:19
↓ 9 callersFunctionsecurity_path
``value`` as a lowercase normalized path: ``~``-anchored inside the agent's home, else absolute. Returns ``""`` for an empty word, an option, or
src/skillevaluator/tier3/eval_core/plugin_security.py:2410
↓ 9 callersFunctionsigned_receipt
(run, **changes)
tests/ci/test_verify_provenance.py:33
↓ 9 callersMethodstop
Tear down the live display (safe on errors / KeyboardInterrupt).
src/skillevaluator/reporting/console_ui.py:667
↓ 9 callersMethodstop
(self)
tests/test_tier3_progress.py:60
↓ 9 callersFunctionsummarize_findings
Group raw findings by stable identity and retain every occurrence.
src/skillevaluator/reporting/finding_summary.py:116
↓ 9 callersFunctionsummarize_plugin_signals
Aggregate per-trial ``plugin_signals`` into one per-arm ``plugin_signals_summary``. ``None`` entries are trials whose trajectory could not be rea
src/skillevaluator/tier3/eval_core/plugin_signals.py:5012
↓ 9 callersMethodtake
(self, token: str)
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:919
↓ 9 callersFunctiontext_content
Read rendered fragment text without treating quoted attribute angles as tags.
tests/reporting/report_markup_assertions.py:66
↓ 9 callersFunctiontier3_result
()
tests/reporting/test_retained_tiers.py:19
↓ 9 callersMethodvalidate
Run dependency audit on skill(s) at path.
src/skillevaluator/validators/dependencies.py:125
↓ 9 callersMethodvalidate
Run hygiene checks on skill(s) at path.
src/skillevaluator/validators/hygiene.py:40
↓ 9 callersFunctionvalidate_case_ids
Validate and canonicalize a complete dataset's IDs before mutation.
src/skillevaluator/tier3/case_ids.py:53
↓ 9 callersFunctionvalidate_manifest_fields
Return the field problems of one parsed contained manifest (Claude Code, Agent Plugins, Codex, Cursor, OpenClaw). Bundle-reference manifests keep
src/skillevaluator/plugin_format_schema.py:1676
↓ 9 callersFunctionversion_comparison_lines
Small shared report section, independent of no-skill lift.
src/skillevaluator/tier3/version_comparison.py:245
↓ 9 callersFunctionwrite_output_file_atomically
Durably replace one evaluator-owned output file with complete bytes.
src/skillevaluator/tier3/output_provenance.py:144
↓ 8 callersFunction_absolute_lexical
(path: Path | str)
src/skillevaluator/tier3/harbor/secure_copy.py:190
↓ 8 callersFunction_ae1
(result: ValidationResult)
tests/validators/test_skillspector_reference_diagnostics.py:75
↓ 8 callersFunction_agent_eval_count
(value: object)
src/skillevaluator/reporting/base.py:612
↓ 8 callersFunction_agent_result
()
tests/reporting/test_native_report_parity.py:14
↓ 8 callersFunction_agents
(ae: dict[str, Any] | None)
src/skillevaluator/reporting/benchmark.py:1421
↓ 8 callersFunction_as_result
( name: str, description: str, validator: ValidatorRunner, target: Path, *, publicatio
src/skillevaluator/tier1/commands.py:226
↓ 8 callersFunction_build_conclusions
Generate stable Insights conclusions from canonical scores (SkillEvaluator parity).
src/skillevaluator/evaluation/tier3_report.py:4422
↓ 8 callersFunction_build_metadata_blocking_network_policy
Build a pod-scoped egress NetworkPolicy blocking GCE/GKE metadata HTTP while preserving DNS.
src/skillevaluator/tier3/harbor/gke_environment.py:495
↓ 8 callersFunction_case_id
(value: Any)
src/skillevaluator/tier3/harbor/stats.py:119
↓ 8 callersMethod_check_dead_links
Verify all relative markdown links point to existing files. Markdown files under Tier 1 artifact directories (``evals/``, ``results/`
src/skillevaluator/validators/hygiene.py:56
↓ 8 callersFunction_checks
(findings, severity: Severity | None = None)
tests/validators/test_openclaw_mcp.py:45
↓ 8 callersFunction_clean_text
(value: str)
src/skillevaluator/reporting/plugin_static.py:60
↓ 8 callersFunction_codex_step
(call_id: str, tool: str, args: dict[str, Any], text: str)
tests/tier3/test_f90_mcp_execution.py:147
↓ 8 callersFunction_complete_stats
(**overrides: object)
tests/test_harbor_runner_status.py:2650
↓ 8 callersFunction_completed_run
A completed run whose case passes without the skill and fails with it; no report.html.
tests/tier3/test_view_results_feedback.py:73
↓ 8 callersFunction_condition
(*, cases: int, planned: int, scored: int, judge_failures: int = 0)
tests/reporting/test_evidence_volume_rendering.py:21
↓ 8 callersFunction_detect_from_nested_structure
Detect content type from nested directory structure.
src/skillevaluator/cli_core.py:82
↓ 8 callersFunction_dict
(value: object)
ci/report_views.py:25
↓ 8 callersFunction_digest
(path: Path | None)
tests/test_quality_config.py:373
↓ 8 callersFunction_evaluate
(check, *, calls=None, response="", roots=None)
tests/tier3/test_declarative_checks.py:20
↓ 8 callersFunction_finding
(**overrides)
tests/reporting/test_finding_verification.py:42
↓ 8 callersFunction_finding
(finding_type="destructive_command", *, target_skill_used_before=True, evidence=None)
tests/tier3/test_harbor_security_attribution.py:22
↓ 8 callersFunction_gate
(manifest: str)
tests/validators/test_skillspector_hook_surface.py:181
↓ 8 callersFunction_http_error
(body: str, *, code: int = 401, reason: str = "Unauthorized")
tests/test_verifier_error_redaction.py:71
↓ 8 callersFunction_inline_visible_lines
Flatten inline Markdown into visible lines while excluding inert markup.
scripts/ci/check_public_benchmarks.py:1362
↓ 8 callersFunction_install_fake_ragas
Install the Ragas 0.4 modules and OpenAI client the verifier imports, with ``metric`` as the scorer.
tests/tier3/test_judge_failure_artifacts.py:445
↓ 8 callersFunction_items
(value: object)
ci/report_views.py:29
↓ 8 callersFunction_llm_fix
One item for checks whose LLM stage could not run: a missing key or a failed provider call.
src/skillevaluator/reporting/console_ui.py:1385
↓ 8 callersFunction_mapping
(value: object)
src/skillevaluator/reporting/benchmark.py:1134
↓ 8 callersFunction_metric_view
(payload)
tests/reporting/test_native_overview_comparison.py:53
↓ 8 callersMethod_normalize_license_id
Normalize license ID for case-insensitive comparison.
src/skillevaluator/validators/license.py:777
↓ 8 callersFunction_npm_commands
Every npm invocation in a ``run:`` block, as its argv tokens. Anything holding the word ``npm`` that does not come out of this as a plain npm
tests/test_ci_workflows.py:724
← previousnext →1,001–1,100 of 24,741, ranked by callers