Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/NVIDIA/SkillEvaluator
/ functions
Functions
24,741 in github.com/NVIDIA/SkillEvaluator
⨍
Functions
24,741
◇
Types & classes
1,292
↳
Endpoints
703
↓ 4 callers
Method
emit
(self, event: ProgressEvent)
src/skillevaluator/tier3/harbor/progress.py:143
↓ 4 callers
Function
enabled_check_lineup
Return the resolved check names for a run, in canonical pipeline order. Unrecognized names are kept (sorted, at the end) so the printed lineup
src/skillevaluator/tier1/commands.py:298
↓ 4 callers
Method
evaluate
Run a live agent evaluation and return the engine results mapping.
src/skillevaluator/evaluation/service.py:30
↓ 4 callers
Function
evaluated_source_revision
Return the immutable revision of the evaluated source. Issue #72 accepts either an evaluated source commit SHA *or* a canonical digest of the
src/skillevaluator/source_identity.py:141
↓ 4 callers
Function
evidence_fragment
(evidence, slot)
tests/reporting/test_plugin_evidence_report.py:331
↓ 4 callers
Function
evidence_volume
Build a bounded evidence summary without changing any scores.
src/skillevaluator/tier3/evidence_volume.py:13
↓ 4 callers
Function
expression
Evaluate the small GitHub expression subset these job conditions use.
tests/ci/test_publisher_workflow.py:200
↓ 4 callers
Function
extract_from_workflow
Extract title + description from a workflow's workflow-rules.mdc.
src/skillevaluator/embedding/extractor.py:218
↓ 4 callers
Method
feed_data
(self, data: bytes)
tests/test_harbor_secure_docker_environment.py:188
↓ 4 callers
Method
feed_eof
(self)
tests/test_harbor_secure_docker_environment.py:191
↓ 4 callers
Method
file
(self, path, ref)
src/skillevaluator/_ci_onboarding_all_tiers.py:274
↓ 4 callers
Function
find_claude_marketplace
The Claude Code marketplace that lists the plugin at ``plugin_root``, or ``None``. Claude Code resolves a bare dependency name against the market
src/skillevaluator/plugin_manifest.py:664
↓ 4 callers
Function
find_eval_file
Find an eval file for a skill, checking both new and legacy locations. Search order: 1. ``skill_dir/evals/evals.{json,jsonl,yaml,yml}`` (ne
src/skillevaluator/tier3/dataset_utils.py:27
↓ 4 callers
Function
find_runtime_command
Find the executable for a local-mode agent after prepending runtime bins.
src/skillevaluator/tier3/harbor/local_runtime.py:208
↓ 4 callers
Function
finding_counts
Return a summary's reconciliation counts without its finding groups.
src/skillevaluator/reporting/finding_summary.py:204
↓ 4 callers
Method
finish
(self)
src/skillevaluator/reporting/native_evidence.py:163
↓ 4 callers
Function
fixture
()
tests/reporting/test_native_evidence.py:19
↓ 4 callers
Method
from_entries
(cls, entries: Iterable[str])
src/skillevaluator/validators/mcp_static.py:2986
↓ 4 callers
Function
gate
()
tests/ci/test_gitlab_admission_transport.py:16
↓ 4 callers
Method
getDecimalForPixel
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 4 callers
Method
getIndexAngle
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 4 callers
Method
getOptionScopes
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 4 callers
Function
gym_eval_command
Build the ``gym eval run`` command for one agent and arm; the API key is read from the environment.
src/skillevaluator/tier3/nemo_gym/runner.py:378
↓ 4 callers
Function
harbor_environment_type
Return the Harbor ``--env`` type that implements a SkillEvaluator mode.
src/skillevaluator/tier3_environments.py:72
↓ 4 callers
Function
harness_block
Collect the per-skill harness rows recorded by the ``harness`` check.
src/skillevaluator/harness/report.py:46
↓ 4 callers
Function
harness_note
One-line verdict note naming the harnesses that drop or degrade the skill(s).
src/skillevaluator/harness/report.py:54
↓ 4 callers
Function
has_control_character
The one control-character rule for configured roots and Git paths, DEL included.
ci/select_changed_skills.py:63
↓ 4 callers
Method
hash_staged_arm
``{piece id: sha256}`` of every piece one arm staged into ``env_dir`` (an absent piece is left out).
src/skillevaluator/tier3/loose_parts.py:379
↓ 4 callers
Function
ia
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 4 callers
Method
info
The staging's ``loose_parts`` info block with the emitted tasks' hashes and a load-evidence summary.
src/skillevaluator/tier3/harbor/loose_parts_arm.py:72
↓ 4 callers
Method
initialize
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 4 callers
Function
interaction_scores
Calculate the signed interaction from comparable outcome means.
src/skillevaluator/tier3/interactions.py:37
↓ 4 callers
Method
isPointInArea
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 4 callers
Function
is_credential_key
Whether a key names a credential (``API_KEY``, ``clientSecret``, ``--auth``), not a setting about one. A key whose last word makes it a setting (
src/skillevaluator/validators/url_policy.py:262
↓ 4 callers
Function
is_openai_reasoning_model
Return whether ``model`` is a GPT-5 or GPT-6 reasoning model.
src/skillevaluator/provider_config.py:162
↓ 4 callers
Function
is_operator_owned_or_provider_secret
Return True if an environment variable name belongs to operator, provider, or cloud credentials.
src/skillevaluator/tier3/harbor/runner.py:1325
↓ 4 callers
Function
is_tier2_result
Return whether a result belongs to semantic Tier 2 validation.
src/skillevaluator/reporting/base.py:2040
↓ 4 callers
Function
is_tier3_platform_related
Return whether a change can alter the result of the Tier 3 platform jobs. The answer is ``True`` unless every changed path is known not to matter
scripts/classify_ci_changes.py:86
↓ 4 callers
Method
items
(self, raw: object, *, limit: int = 32)
src/skillevaluator/reporting/plugin_runtime.py:75
↓ 4 callers
Function
job_context
(committed, event="workflow_run", attempt="1", deploy="success", skipped="")
tests/ci/test_publisher_workflow.py:219
↓ 4 callers
Function
k
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 4 callers
Function
lift_style
Style the displayed two-decimal lift; rounded zero is neutral.
src/skillevaluator/reporting/terminal_styles.py:27
↓ 4 callers
Function
load_case_findings
Read the optional advisory cache without following artifact links.
src/skillevaluator/tier3/case_summary.py:892
↓ 4 callers
Function
load_failures_by_case
``{case id: [{arm, attempt, piece, reason}]}`` for every case where a staged piece did not load. ``evidence_by_arm`` maps an arm to ``{case id: [
src/skillevaluator/tier3/loose_parts.py:1663
↓ 4 callers
Function
load_pii_patterns
Load PII detection patterns from YAML config.
src/skillevaluator/config/__init__.py:13
↓ 4 callers
Function
load_prices
(tmp_path)
tests/tier3/test_price_provenance.py:12
↓ 4 callers
Function
managed_runtime_prefix
The npm prefix that holds ``agent``'s managed CLI.
src/skillevaluator/tier3/harbor/local_runtime.py:517
↓ 4 callers
Function
managed_skillspector_wheel
Where the pinned SkillSpector wheel lives; None when the runtime root is invalid.
src/skillevaluator/utils/scanner_install.py:89
↓ 4 callers
Function
manifest
(root: Path, refs: list, *, name="agent_plugin.yaml")
tests/test_plugin_dependency_records.py:24
↓ 4 callers
Function
mark_not_applicable
Record *metric* as N/A on *reward* using the verifier's marker shape. A reason the reward already records for its N/A metric is kept. Otherwise
src/skillevaluator/tier3/harbor/metrics.py:412
↓ 4 callers
Function
mask
(masked_reading: str)
tests/tier3/test_security_finding_evidence.py:1555
↓ 4 callers
Function
merge_completeness
Keep both scan stages and skill identities when result objects are merged.
src/skillevaluator/skillspector_diagnostics.py:79
↓ 4 callers
Function
merge_evaluated_sources
Merge every populated carrier of the identity into one, or raise. A card can carry the identity in more than two places -- the Tier 3 payload,
src/skillevaluator/source_identity.py:159
↓ 4 callers
Function
metric_set_for_rewards
Return the metric set for a collection, preferring the new SkillEvaluator set.
src/skillevaluator/tier3/harbor/metrics.py:493
↓ 4 callers
Function
model_url_credential_error
Return an error when a policy URL carries userinfo, which would land on Gym's command line.
src/skillevaluator/tier3/nemo_gym/backend.py:288
↓ 4 callers
Function
native_archive
(tmp_path, *, tamper=False, conflict=False)
tests/ci/test_benchmark_publish.py:213
↓ 4 callers
Method
normalize
(path: object)
tests/validators/test_skillspector_reference_diagnostics.py:632
↓ 4 callers
Function
normalize_backend
Return the canonical backend name or raise ``ValueError``.
src/skillevaluator/tier3/nemo_gym/backend.py:46
↓ 4 callers
Function
not_supported_reason
Why an agent other than Claude Code has no loose-parts arm in v1.
src/skillevaluator/tier3/loose_parts.py:249
↓ 4 callers
Method
note
(self, value: object)
src/skillevaluator/reporting/native_evidence.py:136
↓ 4 callers
Function
npm_clean
()
tests/validators/test_private_dependency_audit.py:65
↓ 4 callers
Method
ok_response
Build an HTTP 200 Converse output response.
tests/test_retry.py:448
↓ 4 callers
Function
open_feedback_session
Load a run's evidence and start a feedback session. The provider configuration is checked here, but no request is sent until the first questi
src/skillevaluator/evaluation/tier3_feedback_chat.py:1065
↓ 4 callers
Function
pair_metric_scores
Score both arms per metric on the cases both scored it on. *treatment* and *control* are ``(case id, logical reward)`` rows; *arms* names the
src/skillevaluator/tier3/harbor/lift_basis.py:77
↓ 4 callers
Function
parse_optional_bool_flag
Parse an optional boolean setting (1/true/yes vs 0/false/no), returning None when unset.
src/skillevaluator/tier3/harbor/gke_environment.py:347
↓ 4 callers
Function
payload
()
tests/reporting/test_private_runtime_render.py:55
↓ 4 callers
Method
peek
(self)
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:915
↓ 4 callers
Function
plugin_slug
A Claude Code-safe plugin name: kebab-case, no spaces, ``@``, ``:``, or separators.
src/skillevaluator/tier3/plugin_native.py:201
↓ 4 callers
Method
preflight
(cls)
tests/test_custom_execution_backends.py:38
↓ 4 callers
Method
price_for
(self, model: str)
src/skillevaluator/tier3/harbor/run_budget.py:87
↓ 4 callers
Function
probe_mcp_servers
Probe every author-supplied URL MCP server (bounded to 32 servers).
src/skillevaluator/tier3/mcp_proof.py:768
↓ 4 callers
Function
project_plugin_runtime
Copy recorded observations into bounded views without rescoring or I/O.
src/skillevaluator/reporting/plugin_runtime.py:386
↓ 4 callers
Method
provenance
(self)
src/skillevaluator/tier3/plugin_comparisons.py:487
↓ 4 callers
Function
publication_target_conflict_marker
Flatten an untrusted conflict claim to one bounded, printable line.
src/skillevaluator/reporting/base.py:854
↓ 4 callers
Function
publication_target_name
Return the NFC entry name that publication targets record for a path. On a case- or normalization-insensitive filesystem a lexical alias such as
src/skillevaluator/publication_identity.py:678
↓ 4 callers
Function
publish
Append an immutable report and advance only API-verified current aliases.
ci/publish_reports.py:971
↓ 4 callers
Method
put
(self, path, payload)
ci/feedback.py:628
↓ 4 callers
Method
read_bounded
Open one contained regular file and perform exactly one bounded read.
src/skillevaluator/deduplication/utils/skill_collector.py:154
↓ 4 callers
Function
read_json
(path: Path)
ci/run_tiers.py:167
↓ 4 callers
Method
recalculate_keeping_independent
Rebuild from findings, then put back the errors and warnings that no finding repeats. :meth:`recalculate_from_findings` drops every error and
src/skillevaluator/models/result.py:600
↓ 4 callers
Function
recorded_evaluated_source
Return the single identity a run recorded, or raise if it recorded two. Every carrier of every result is folded, rather than one nested agent-eva
src/skillevaluator/source_identity.py:214
↓ 4 callers
Function
recovered_run_report
Collect a timeout on attempt 1 and a pass on attempt 2 of 3, and load it for the HTML reporter.
tests/test_harbor_timeout_recovery.py:398
↓ 4 callers
Function
redacted_url
URL for finding messages: scheme, host, port, and path only. Userinfo, parameters, query, and fragment are dropped so an inline credential is
src/skillevaluator/validators/mcp_static.py:287
↓ 4 callers
Function
registry
(monkeypatch, outcome)
tests/ci/test_release_image.py:29
↓ 4 callers
Function
render
(tmp_path, run)
tests/reporting/test_resource_comparison.py:111
↓ 4 callers
Method
render
Render single result to JSON.
src/skillevaluator/reporting/json_reporter.py:135
↓ 4 callers
Method
render
Render single result to Markdown.
src/skillevaluator/reporting/markdown.py:210
↓ 4 callers
Method
render_all
(self, results: list[ValidationResult])
src/skillevaluator/reporting/sarif_reporter.py:353
↓ 4 callers
Function
render_external_metrics_markdown
(report: dict[str, Any])
src/skillevaluator/reporting/external_metrics.py:25
↓ 4 callers
Method
render_result
Render a single validation result.
src/skillevaluator/reporting/cli.py:274
↓ 4 callers
Function
replace
(match: re.Match[str])
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:360
↓ 4 callers
Function
report
(*, success=True, errors=None, warnings=None)
tests/validators/test_private_format_boundaries.py:44
↓ 4 callers
Function
report_url
(name: str, custom: str | None = None, *, verified_private_url: str | None = None)
src/skillevaluator/ci_onboarding.py:143
↓ 4 callers
Function
result_execution_status
Display explicit N/A evidence without changing gate or JSON status.
src/skillevaluator/reporting/presentation.py:157
↓ 4 callers
Function
rewards_have_mixed_metric_contracts
Return whether physical reward rows declare distinct metric contracts.
src/skillevaluator/tier3/harbor/metrics.py:481
↓ 4 callers
Function
rollout_task_name
Return the task folder name a Gym Harbor rollout ran.
src/skillevaluator/tier3/nemo_gym/results.py:79
↓ 4 callers
Method
run
(self, instruction, environment: BaseEnvironment, context)
src/skillevaluator/tier3/harbor/local_agents.py:604
↓ 4 callers
Function
safe
(value: object)
src/skillevaluator/tier3/case_summary.py:789
↓ 4 callers
Function
sanitize_completeness
Re-project saved metadata at report boundaries and preserve omission counts.
src/skillevaluator/skillspector_diagnostics.py:59
↓ 4 callers
Function
sanitize_path_text
Replace absolute forms of supplied paths with stable, relative labels.
src/skillevaluator/utils/tier2_paths.py:122
← previous
next →
2,401–2,500 of 24,741, ranked by callers