Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/NVIDIA/SkillEvaluator
/ types & classes
Types & classes
1,292 in github.com/NVIDIA/SkillEvaluator
⨍
Functions
24,741
◇
Types & classes
1,292
↳
Endpoints
703
↓ 669 callers
Class
ValidationResult
Aggregates validation outcomes from a validator. This is the primary data structure passed to reporters. It contains both structured findings
src/skillevaluator/models/result.py:392
↓ 315 callers
Class
HTMLReporter
HTML report generator using Jinja2 templates.
src/skillevaluator/reporting/html.py:2196
↓ 287 callers
Class
SecurityValidator
Scans skills for security vulnerabilities and PII leakage. Combines the skillspector CLI (15+ vulnerability patterns) with custom PII detecti
src/skillevaluator/validators/security.py:2960
↓ 244 callers
Class
JSONReporter
JSON export for machine-readable output. Produces structured JSON output suitable for programmatic consumption. Supports both compact and pre
src/skillevaluator/reporting/json_reporter.py:93
↓ 222 callers
Class
BenchmarkReporter
Render a stable, publication-oriented ``BENCHMARK.md`` card.
src/skillevaluator/reporting/benchmark.py:156
↓ 209 callers
Class
Finding
Structured validation issue with full context for CI reporting. Provides detailed information about each validation issue to help developers
src/skillevaluator/models/result.py:83
↓ 164 callers
Class
ProviderConfig
Resolved provider values safe to pass to the relevant SDK.
src/skillevaluator/provider_config.py:220
↓ 115 callers
Class
MarkdownReporter
Markdown report generator for PR comments. Produces clean, readable Markdown suitable for GitHub pull request comments and documentation.
src/skillevaluator/reporting/markdown.py:175
↓ 108 callers
Class
ToolResult
Result from running an external tool. ``success`` only means the process launched and ran to completion (no missing binary, timeout, or OS er
src/skillevaluator/utils/tool_runner.py:103
↓ 96 callers
Class
SchemaValidator
Validates skill schema and repository structure per SkillEvaluator spec. Checks: frontmatter schema, folder hierarchy, naming conventions, di
src/skillevaluator/validators/schema.py:224
↓ 77 callers
Class
EmbeddingRegistry
Builds and queries an in-memory embedding index. Supports two workflows: 1. Live scan: build_from_directory() discovers content, embeds it, s
src/skillevaluator/embedding/registry.py:188
↓ 76 callers
Class
LLMClient
Public-provider client for chat completions. Supports two usage modes: 1. **Direct** -- call :meth:`completions` or :meth:`extract_js
src/skillevaluator/inference/client.py:439
↓ 71 callers
Class
CLIReporter
Terminal-based reporter with Rich formatting. Provides colorful, structured output for terminal/console display. Supports both direct printin
src/skillevaluator/reporting/cli.py:122
↓ 56 callers
Class
IntraSkillValidator
Detect redundant content within a skill directory.
src/skillevaluator/deduplication/intra_skill/intra_skill_validator.py:161
↓ 56 callers
Class
QualityScoreValidator
Evaluates skill quality across 4 weighted dimensions. Produces a 0-100 composite score with A-F letter grades. Designed to be compatible with
src/skillevaluator/validators/quality_score.py:246
↓ 55 callers
Class
SecureRoot
Descriptor-anchored reads beneath one verified regular root.
src/skillevaluator/utils/secure_fs.py:300
↓ 53 callers
Class
LicenseValidator
Validates license compliance for Skills, Rules, and Workflows. Uses a multi-tier detection approach to find license information and validates
src/skillevaluator/validators/license.py:180
↓ 50 callers
Class
BudgetController
src/skillevaluator/tier3/harbor/run_budget.py:117
↓ 49 callers
Class
UnsafeStagingError
Raised when authored or destination content is unsafe to stage.
src/skillevaluator/tier3/harbor/secure_copy.py:80
↓ 48 callers
Class
ExternalTool
Wrapper for external CLI tools with consistent error handling. Provides automatic PATH lookup, installation hints, timeout handling, and stru
src/skillevaluator/utils/tool_runner.py:141
↓ 45 callers
Class
EvaluationOptions
Options for a Tier 3 live agent evaluation run. Field names mirror :func:`skillevaluator.tier3.commands.evaluate` keyword arguments so the se
src/skillevaluator/evaluation/options.py:19
↓ 44 callers
Class
DependencySecurityValidator
Scan dependencies for CVE vulnerabilities. Uses pip-audit (primary) and Safety (secondary) to detect known security issues in Python dependen
src/skillevaluator/validators/dependencies.py:94
↓ 43 callers
Class
SetupError
A safely described setup failure.
src/skillevaluator/_ci_onboarding_git.py:32
↓ 42 callers
Class
SecurePathError
An unsafe, racy, inaccessible, or unbounded filesystem input.
src/skillevaluator/utils/secure_fs.py:67
↓ 39 callers
Class
SimilarityValidator
Detect duplicate content via embedding similarity. Supports all three SkillEvaluator content types (skill, rules, workflows). Produces struct
src/skillevaluator/validators/similarity.py:33
↓ 38 callers
Class
CodeRiskValidator
Static code analysis for security risks. Combines Bandit's Python checks with a packaged, high-confidence Semgrep baseline for command/dynami
src/skillevaluator/validators/code_risk.py:82
↓ 37 callers
Class
ValidationPolicy
Validation policy bundle consumed by validators. Attributes: profile: Human-readable profile name or a custom name supplied by a
src/skillevaluator/validators/policy.py:57
↓ 36 callers
Class
RubricEvalValidator
LLM-as-Judge rubric evaluation for skill quality. Sends the full SKILL.md to an LLM that scores it against 9 qualitative criteria. Produces a
src/skillevaluator/validators/rubric_eval.py:611
↓ 34 callers
Class
InsightsJudge
LLM judge that produces additional conclusions and recommendations.
src/skillevaluator/evaluation/insights_judge.py:535
↓ 29 callers
Class
BridgePayloadError
A malformed bridge payload that an HTTP layer should return as 400.
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:130
↓ 29 callers
Class
HygieneValidator
Validates code integrity: dead links, dependencies, and test-file presence.
src/skillevaluator/validators/hygiene.py:29
↓ 28 callers
Class
ContentCluster
A group of semantically similar chunks.
src/skillevaluator/deduplication/intra_skill/semantic_clustering.py:28
↓ 28 callers
Class
LLMVerdict
Structured result from LLM verification of a content cluster.
src/skillevaluator/inference/types.py:44
↓ 27 callers
Class
EmbeddingClient
Public OpenAI-compatible client for generating text embeddings. Uses the OpenAI SDK because NVIDIA Build exposes an OpenAI-compatible API. Cl
src/skillevaluator/embedding/client.py:97
↓ 26 callers
Class
JUnitReporter
JUnit XML export for CI test-report integrations.
src/skillevaluator/reporting/junit_reporter.py:425
↓ 26 callers
Class
PublicationError
Publication failed without exposing tokens or untrusted diagnostics.
ci/publish_reports.py:57
↓ 26 callers
Class
UnicodeSmuggleValidator
Detects invisible Unicode characters that may indicate ASCII smuggling, hidden data encoding, or trojan source attacks.
src/skillevaluator/validators/unicode_smuggle.py:47
↓ 25 callers
Class
PluginRootReader
No-follow classification and bounded reads beneath one plugin root. Security-relevant JSON configs (hooks, LSP, monitors, settings, MCP) draw on
src/skillevaluator/plugin_paths.py:124
↓ 24 callers
Class
ClaudeCodeAdapter
Claude Code: a session plugin loaded with the documented ``--plugin-dir`` flag.
src/skillevaluator/tier3/plugin_native.py:1350
↓ 24 callers
Class
PluginMember
One declaration or client-visible packaged member, without policy or runtime conclusions.
src/skillevaluator/plugin_discovery.py:80
↓ 24 callers
Class
_RecordingReporter
tests/test_tier3_progress.py:2812
↓ 23 callers
Class
SecretsValidator
Scan for hardcoded secrets using Gitleaks. Provides entropy-based detection, 200+ pre-built patterns, optional git history scanning, and low
src/skillevaluator/validators/secrets.py:35
↓ 23 callers
Class
_TTYBuffer
tests/test_tier3_progress.py:35
↓ 22 callers
Class
ModelRecord
One normalized model returned by a provider catalog.
src/skillevaluator/model_catalog.py:340
↓ 22 callers
Class
RubricJudge
LLM judge that scores a skill against qualitative criteria.
src/skillevaluator/validators/rubric_eval.py:268
↓ 21 callers
Class
PrivatePluginChecks
src/skillevaluator/tier1/plugin_checks.py:39
↓ 21 callers
Class
ScriptLintValidator
Advisory AST-based linter for skill scripts/ directories. All findings are MEDIUM/LOW severity — they produce warnings but never fail validat
src/skillevaluator/validators/script_lint.py:93
↓ 20 callers
Class
ContentChunk
A section or paragraph extracted from a file within a skill.
src/skillevaluator/deduplication/utils/chunker.py:37
↓ 20 callers
Class
EvalsConfigError
Raised when ``evals/config.yml`` is present but invalid.
src/skillevaluator/tier3/evals_config.py:137
↓ 20 callers
Class
PluginSchemaValidator
Validate a plugin manifest and any skills bundled by the plugin. Checks: manifest presence, parseable YAML, and the :class:`PluginManifest` c
src/skillevaluator/validators/plugin_schema.py:42
↓ 19 callers
Class
SARIFReporter
SARIF 2.1.0 export for security scanning integrations.
src/skillevaluator/reporting/sarif_reporter.py:326
↓ 19 callers
Class
VersionValidator
Validate optional skill semantic version labels.
src/skillevaluator/validators/version.py:54
↓ 18 callers
Class
FakeTool
tests/validators/test_private_dependency_audit.py:32
↓ 18 callers
Class
ProgressEvent
A structured progress transition emitted by the Tier 3 engine.
src/skillevaluator/tier3/harbor/progress.py:121
↓ 17 callers
Class
EvaluationService
Facade over the native Tier 3 engine used by both the CLI and the API.
src/skillevaluator/evaluation/service.py:27
↓ 17 callers
Class
LLMClientError
Raised when an LLM operation fails (missing key, bad response, etc.).
src/skillevaluator/inference/types.py:15
↓ 16 callers
Class
CheckResult
src/skillevaluator/tier3/evals_spec.py:345
↓ 16 callers
Class
FakeEmbedding
tests/deduplication/plugin/test_private_dedup.py:32
↓ 16 callers
Class
PluginManifest
Top-level bundle-reference plugin manifest (``agent_plugin.yaml``). Unknown top-level fields are rejected. ``name`` and ``author`` (with a no
src/skillevaluator/models/plugin.py:181
↓ 16 callers
Class
Redactor
Compile the token set once for efficient, longest-first replacement.
src/skillevaluator/utils/secret_redaction.py:132
↓ 16 callers
Class
ValidateView
Progressive pipeline renderer for validate. Drive it from the command: :meth:`start`, then :meth:`tier_start` / :meth:`tier_progress` / :meth
src/skillevaluator/reporting/console_ui.py:527
↓ 15 callers
Class
FakeAPI
tests/ci/test_publish_reports.py:448
↓ 15 callers
Class
PreparationError
The inventory cannot be prepared for a runtime evaluation.
src/skillevaluator/tier3/plugin_prepare.py:60
↓ 15 callers
Class
RegistryEntry
A single item in the embedding index.
src/skillevaluator/embedding/registry.py:120
↓ 15 callers
Class
TierRow
One detail line inside a tier section.
src/skillevaluator/reporting/console_ui.py:258
↓ 15 callers
Class
TrialObservation
One logical Harbor attempt: its case, reward payload and resource usage. ``usage`` holds optional non-negative counters read from the trial's ATI
src/skillevaluator/tier3/harbor/stats.py:79
↓ 15 callers
Class
_StreamedComposeProcess
tests/test_harbor_secure_docker_environment.py:244
↓ 14 callers
Class
CensusCheck
One post-setup listing check that the in-container census script runs. ``kind`` is ``file`` (a regular file exists), ``dir`` (a directory exists)
src/skillevaluator/tier3/plugin_native.py:805
↓ 14 callers
Class
ConfigurationError
A fixed diagnostic that contains no credential or untrusted command output.
ci/run_tiers.py:73
↓ 14 callers
Class
Environment
tests/test_harbor_local_mode.py:715
↓ 14 callers
Class
ModelProbeResult
Safe result from a provider model-catalog request.
src/skillevaluator/tier3/harbor/runtime_preflight.py:108
↓ 14 callers
Class
PluginLoadError
Raised when ``--plugin-load native`` cannot be honored for a selected agent.
src/skillevaluator/tier3/plugin_native.py:112
↓ 14 callers
Class
Tier3FeedbackSession
Multi-turn conversation over one run's evidence. The evidence travels in the system message, so the history alternates user and assistant tur
src/skillevaluator/evaluation/tier3_feedback_chat.py:1007
↓ 14 callers
Class
_BootstrapEnvironment
tests/tier3/test_installed_agent_bootstrap.py:42
↓ 14 callers
Class
_ReportBudget
src/skillevaluator/evaluation/tier3_report.py:175
↓ 14 callers
Class
_TrajectoryMergeError
A multi-file ATIF trajectory cannot be materialized without data loss.
src/skillevaluator/tier3/harbor/collector.py:1868
↓ 13 callers
Class
ClaudePluginValidateParity
Run ``claude plugin validate`` and compare it with SkillEvaluator's manifest verdict and findings.
src/skillevaluator/validators/claude_plugin_validate.py:383
↓ 13 callers
Class
ConcreteValidator
Concrete implementation of ValidatorBase for testing.
tests/validators/test_base.py:88
↓ 13 callers
Class
FindingVerifier
Classifies security / PII findings as true or false positives.
src/skillevaluator/inference/finding_verifier.py:30
↓ 13 callers
Class
HookAnalyzer
Analyze every handler of a plugin's hooks config sources. ``read_script`` returns the whole text of a script under the plugin root (at most `
src/skillevaluator/plugin_component_risk.py:3052
↓ 13 callers
Class
SimilarityConfigError
Raised when configuration or provider failures prevent embedding comparison.
src/skillevaluator/embedding/client.py:35
↓ 12 callers
Class
ContentEntry
Unified representation of a content item for embedding. Attributes: name: Skill name or rule/workflow title. description: Descrip
src/skillevaluator/embedding/extractor.py:72
↓ 12 callers
Class
FakeAPI
tests/ci/test_comment_report.py:45
↓ 12 callers
Class
Offender
scripts/ci/check_public_benchmarks.py:742
↓ 12 callers
Class
ProviderConfigurationError
Raised when a selected public provider is not fully configured.
src/skillevaluator/provider_config.py:215
↓ 12 callers
Class
RulesSchemaValidator
Validates Rules .mdc schema and repository structure per SkillEvaluator spec. Checks: frontmatter schema, folder hierarchy, naming conventions,
src/skillevaluator/validators/rules_schema.py:34
↓ 12 callers
Class
StreamingLogRedactor
Redact known key shapes and exact values across arbitrary chunks. Each family runs once on either side of the other. The post-known pass cat
src/skillevaluator/tier3/harbor/stream_redaction.py:272
↓ 12 callers
Class
WorkflowsSchemaValidator
Validates Workflows directory structure and content per SkillEvaluator spec. Checks: - Directory structure (README.md, workflow-rules.mdc, re
src/skillevaluator/validators/workflows_schema.py:45
↓ 11 callers
Class
FeedbackError
Fixed diagnostic containing no credentials or untrusted response text.
ci/feedback.py:83
↓ 11 callers
Class
ModelCatalogError
Safe-to-display catalog error with non-secret structured metadata.
src/skillevaluator/model_catalog.py:70
↓ 11 callers
Class
_BufferedAndStreamedComposeProcess
tests/test_harbor_secure_docker_environment.py:271
↓ 11 callers
Class
_BufferedComposeProcess
tests/test_harbor_secure_docker_environment.py:112
↓ 11 callers
Class
_NotStatic
The wrapper source is outside the extended static grammar.
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:891
↓ 10 callers
Class
McpPinning
Static supply-chain pinning classification of one MCP declaration. ``floating`` marks an ``unpinned`` launch whose version or tag names a mov
src/skillevaluator/validators/mcp_static.py:1907
↓ 10 callers
Class
Requirement
src/skillevaluator/tier3/evals_spec.py:41
↓ 10 callers
Class
SkillEvaluatorClaudeCode
Wrap Claude Code to preserve rich MCP server declarations and Vertex AI routing. It extends the installed-agent wrapper, so Harbor's installer st
src/skillevaluator/tier3/harbor/local_agents.py:173
↓ 10 callers
Class
TrialTraceArtifacts
Read one whole-trial trajectory and harness logs once per logical root.
src/skillevaluator/tier3/plugin_trace_artifacts.py:502
↓ 9 callers
Class
DeclaredPath
src/skillevaluator/plugin_paths.py:66
↓ 9 callers
Class
FeedbackError
A fixed diagnostic safe to print without feedback content or credentials.
ci/comment_report.py:34
↓ 9 callers
Class
MockUrllibResponse
Mock a urllib response context manager returning serialized JSON or raw bytes.
tests/conftest.py:591
next →
1–100 of 1,292, ranked by callers