MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 8 callersFunction_number
(value: object)
src/skillevaluator/reporting/plugin_static.py:38
↓ 8 callersFunction_options
(tmp_path: Path)
tests/tier3/test_plugin_execution.py:30
↓ 8 callersFunction_overall_verdict_from_agents
PASS only when one supported agent passes its quality requirements.
src/skillevaluator/evaluation/tier3_report.py:4690
↓ 8 callersFunction_overview
(html)
tests/reporting/test_native_feature_restoration.py:104
↓ 8 callersFunction_package
(root: Path)
tests/tier3/test_plugin_dataset.py:29
↓ 8 callersFunction_path_problem_finding
HIGH finding for a declared component path that is missing, escapes, or is unsafe.
src/skillevaluator/plugin_paths.py:282
↓ 8 callersFunction_pem_block
( label: str, body: str = "synthetic-key-material-for-redaction-only", *, end_label: str | Non
tests/test_safety_boundaries.py:16
↓ 8 callersFunction_plugin
(tmp_path: Path, files: dict)
tests/tier3/test_plugin_dataset.py:35
↓ 8 callersFunction_probe
(status: str, detail: str = HOST_OK, tools: tuple[str, ...] = ())
tests/tier3/test_f90_mcp_proof_status.py:17
↓ 8 callersFunction_provider_environment
Build evaluator-owned verifier variables from provider config and host overrides.
src/skillevaluator/tier3/harbor/runner.py:2032
↓ 8 callersFunction_read_bounded_text
Read bounded diagnostic text without following links or special files.
src/skillevaluator/tier3/harbor/collector.py:524
↓ 8 callersMethod_read_member
( self, component: PluginMember, rel: PurePosixPath, *, lenient: bool = False, strip: bool = False
src/skillevaluator/plugin_discovery.py:1276
↓ 8 callersFunction_recorded_lines
Return the three identity lines with any of them replaced by ``values``.
tests/test_benchmark_source_provenance.py:549
↓ 8 callersFunction_redact
( text: str | None, secret_values: set[str], *, replacement: str | None = None, include_sh
src/skillevaluator/tier3/harbor/secure_docker_environment.py:629
↓ 8 callersFunction_redact_configured_credentials
(text: str, extra_secret_values: tuple[str | None, ...] = ())
src/skillevaluator/tier3/eval_core/llm_judge.py:124
↓ 8 callersFunction_redact_configured_credentials
(text, extra_secret_values=())
src/skillevaluator/tier3/harbor/templates/eval.py:1160
↓ 8 callersFunction_redacted_artifact_text
(src: Path, text: str)
src/skillevaluator/tier3/harbor/collector.py:1266
↓ 8 callersFunction_render
(format_name: str, result: ValidationResult)
tests/reporting/test_finding_verification.py:62
↓ 8 callersFunction_report
(output_dir)
tests/cli/test_tier_workflows.py:58
↓ 8 callersFunction_resolve_single_kubeconfig
Resolve a single existing kubeconfig file path from KUBECONFIG or default location. If multiple path entries are specified, merge them into a sin
src/skillevaluator/tier3/harbor/kubeconfig.py:76
↓ 8 callersFunction_result
(name)
tests/cli/test_tier_workflows.py:52
↓ 8 callersFunction_run
(tmp_path, *, goal_accuracy: float)
tests/reporting/test_case_findings_report.py:14
↓ 8 callersFunction_run_llm_scan
Run a static stage that completes, then an LLM stage that returns *llm_report*.
tests/validators/test_security.py:6967
↓ 8 callersFunction_run_verdict_lines
(result: dict)
tests/test_tier3_result_display.py:268
↓ 8 callersMethod_safe_text
(self, value: object)
src/skillevaluator/tier3/harbor/progress.py:394
↓ 8 callersFunction_sanitize_staged_runtime_skills
Remove evaluator-only roots from authored or evaluator-copied skill bundles.
src/skillevaluator/tier3/harbor/adapter.py:4174
↓ 8 callersMethod_scanner
(bindir, version_output, version_exit=0)
tests/utils/test_tool_runner.py:485
↓ 8 callersFunction_score_transition_values
Render ``baseline → score (uplift)`` so the visible arithmetic is exact. Scores are fractions in [0, 1]. A missing, non-finite, or out-of-range s
src/skillevaluator/reporting/benchmark.py:1858
↓ 8 callersFunction_security_finding
( *, finding_type: str, severity: str, message: str, evidence: str, source: str, s
src/skillevaluator/tier3/eval_core/checks.py:3285
↓ 8 callersFunction_skill
(name: str)
tests/tier3/test_plugin_dataset.py:17
↓ 8 callersFunction_statuses
(diagnostic, attempt=0)
tests/reporting/test_tier3_expectations.py:40
↓ 8 callersFunction_tier3_harbor_result
Tier 3 result with safe Harbor links and one unsafe link to sanitize.
tests/reporting/test_reporters.py:1882
↓ 8 callersFunction_tier3_identifier_list
Return visible, normalized, collision-free identifiers in source order.
src/skillevaluator/reporting/html.py:447
↓ 8 callersFunction_toml_quote
Return a TOML-compatible quoted string.
src/skillevaluator/tier3/harbor/adapter.py:2294
↓ 8 callersFunction_trajectory_with_late_write
()
tests/tier3/test_behavior_evidence.py:28
↓ 8 callersFunction_unit_embeddings
Return an ``embed`` side effect producing one valid vector per input.
tests/deduplication/intra_skill/test_intra_skill_validator.py:35
↓ 8 callersFunction_use_ragas_eligible_judge
Select a canonical OpenAI non-reasoning judge, which routes goal_accuracy to Ragas.
tests/tier3/test_judge_failure_artifacts.py:411
↓ 8 callersFunction_validate
(target: Path, output: Path, *args: str)
tests/cli/test_validate_manifest_file.py:27
↓ 8 callersFunction_validate_type
( metadata: os.stat_result, *, path: Path, role: str, root_device: int | None, )
src/skillevaluator/tier3/harbor/secure_copy.py:283
↓ 8 callersFunction_write
(root: Path, files: dict[str, str])
tests/validators/test_skillspector_hook_surface.py:189
↓ 8 callersFunction_write_release_review_skill
(tmp_path: Path, assertions: list[str])
tests/tier3/test_dataset_baseline_fairness.py:277
↓ 8 callersFunction_write_skill
(tmp_path: Path, *, native: bool = False)
tests/test_harbor_output_provenance.py:37
↓ 8 callersMethod_write_skill
(root: Path, name: str, id_line: str | None)
tests/validators/test_schema.py:1225
↓ 8 callersFunction_xml_text
Return *value* as XML 1.0 text, dropping characters the spec forbids.
src/skillevaluator/reporting/junit_reporter.py:77
↓ 8 callersFunctionadditional_errors
Return errors not already displayed through a structured finding. Findings also populate the legacy error list. Keep independent errors, incl
src/skillevaluator/reporting/base.py:2562
↓ 8 callersFunctionbuild_final_render_context
Classify once for final human renders; decision scope stays independent.
src/skillevaluator/reporting/presentation.py:272
↓ 8 callersMethodcall_bedrock
Invoke verifier._call_bedrock with default test model parameters.
tests/test_retry.py:456
↓ 8 callersFunctioncanonicalize_trusted_root_alias
Expand a root-owned POSIX alias such as macOS ``/var`` or ``/tmp``. Only the first component is eligible, and only when both the filesystem r
src/skillevaluator/utils/path_security.py:15
↓ 8 callersFunctioncheck_security
(request: pytest.FixtureRequest)
tests/tier3/test_security_home_paths.py:46
↓ 8 callersMethodcollect_arm
Cache one arm once; callers may register later private arms through ``arm_jobs``.
src/skillevaluator/tier3/plugin_evidence.py:411
↓ 8 callersFunctioncomment
(identifier, body, *, login=BOT, user_type="Bot")
tests/ci/test_comment_report.py:36
↓ 8 callersFunctiondataset_coverage_result
Load authored entries; leave invalid and native sources to their validators.
src/skillevaluator/tier3/dataset_coverage.py:69
↓ 8 callersFunctiondataset_provenance_display
(value: object)
src/skillevaluator/tier3/dataset_provenance.py:63
↓ 8 callersMethoddeclared_for
Declared components staged in ``agent``'s ``arm``. MCP servers, subagents, and commands are staged in the with-plugin arm and in the
src/skillevaluator/tier3/eval_core/plugin_signals.py:1337
↓ 8 callersFunctiondimension_scores
Compute report-only SkillEvaluator dimension scores from default metric scores. Source metrics that are N/A for the whole arm (and so absent from
src/skillevaluator/tier3/harbor/metrics.py:717
↓ 8 callersFunctionextract
(_archive: Path, _target: Path, **kwargs: object)
tests/test_harbor_secure_docker_environment.py:4292
↓ 8 callersFunctionextract_http_status
Use the innermost HTTP status along the exception's causal chain. Provider adapters can replace an HTTP status with a generic 500 or 503. Ret
src/skillevaluator/inference/retry.py:124
↓ 8 callersMethodfailure_reason
Return why a direct evaluation result is not a completed run. The standalone command is a gating surface: an empty mapping, a legacy
src/skillevaluator/evaluation/service.py:44
↓ 8 callersMethodfinalize
Apply the requested policy and return actual filesystem truth.
src/skillevaluator/tier3/harbor/artifact_retention.py:39
↓ 8 callersMethodgetUserBounds
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 8 callersFunctiongithub_api
(method: str, endpoint: str, payload: dict | None = None, *, missing_ok=False)
src/skillevaluator/ci_onboarding.py:431
↓ 8 callersFunctionis_sensitive_key
Return whether a structured-data key conventionally carries a secret.
src/skillevaluator/utils/redaction.py:187
↓ 8 callersFunctionkinds
(result)
tests/tier3/test_private_plugin_security.py:34
↓ 8 callersFunctionline
(event: dict)
tests/claude_refusal_fixtures.py:34
↓ 8 callersFunctionload_external_metrics
Validate a bounded, duplicate-free JSON document and hash its exact bytes.
src/skillevaluator/tier3/external_metrics.py:191
↓ 8 callersFunctionloose_parts_load_evidence
``{piece id: {loaded, source, evidence, ...}}`` for one trial of ``arm``. Only Claude Code's own ``system/init`` report marks a piece loaded
src/skillevaluator/tier3/loose_parts.py:1549
↓ 8 callersMethodmanifest
The staged ``plugin.json``: identity plus the component keys that stay native. MCP servers move to the staged ``.mcp.json``, hooks to the cen
src/skillevaluator/tier3/plugin_native.py:1393
↓ 8 callersFunctionnemo_gym_cli_error
Check the task-folder CLI contract without starting servers or passing policy credentials. A version string alone does not identify this interfac
src/skillevaluator/tier3/nemo_gym/backend.py:115
↓ 8 callersFunctionnot_applicable_metrics
Return metrics that every reward records as N/A, so no reward scores them.
src/skillevaluator/tier3/harbor/metrics.py:346
↓ 8 callersMethodoverall_score
(self)
src/skillevaluator/models/quality.py:129
↓ 8 callersFunctionparse_agents
Parse a comma-separated Harbor agent list.
src/skillevaluator/tier3/commands.py:365
↓ 8 callersFunctionprepared
(root)
tests/tier3/test_plugin_evidence.py:20
↓ 8 callersFunctionprevious_source_branch
(trusted, monkeypatch, *, unrelated=False, trailer=None)
tests/ci/test_benchmark_publish.py:577
↓ 8 callersFunctionpublication_confusable_skeleton
Return the pinned UTS #39 skeleton subset needed by reserved identities.
src/skillevaluator/publication_text.py:381
↓ 8 callersMethodreport
Flag the handler (each flag once) and add a finding attributed to it.
src/skillevaluator/plugin_component_risk.py:2860
↓ 8 callersFunctionreported_ref
A declared component reference as findings and the inventory show it, without its credentials. A reference with a URL scheme (``ws://user:token@h
src/skillevaluator/plugin_paths.py:266
↓ 8 callersFunctionrequire_bounded_string
Return a real bounded string without coercing attacker-controlled objects.
src/skillevaluator/utils/structured_data.py:442
↓ 8 callersFunctionresolve_repository_identity
Establish the repository root and its ``origin`` slug for ``plugin_root``. Root precedence: an explicit ``repo_root`` (``--repo-root``) that cont
src/skillevaluator/plugin_dependencies.py:370
↓ 8 callersFunctionrun_similarity_check
( content_path: Path, *, content_type: str = "auto", threshold: float = 0.75, full_body: b
src/skillevaluator/tier2/commands.py:43
↓ 8 callersFunctionsafe_path_label
Return a useful path label without exposing its host directory.
src/skillevaluator/utils/tier2_paths.py:19
↓ 8 callersFunctionscore_definition
Human-readable definition for the SkillEvaluator overall score.
src/skillevaluator/tier3/harbor/metrics.py:649
↓ 8 callersFunctionselect_agent_eval_payload
Select the strongest Tier 3 payload without depending on result order.
src/skillevaluator/reporting/base.py:1866
↓ 8 callersMethodstart
(self)
tests/test_harbor_runtime_preflight.py:3640
↓ 8 callersFunctionsteps_by_case_chart
Return step-chart labels and one dataset per agent and condition. With-skill rows come from the canonical top-level ``trials`` (annotated with
src/skillevaluator/reporting/trial_charts.py:64
↓ 8 callersFunctionsummarize
(payload: dict)
tests/tier3/test_external_metrics.py:64
↓ 8 callersMethodterminate
(self)
tests/reporting/test_dashboard_cli.py:42
↓ 8 callersMethodtier_progress
Update a running tier's detail rows (check ticker / run config).
src/skillevaluator/reporting/console_ui.py:606
↓ 8 callersMethodtier_start
(self, index: int)
src/skillevaluator/reporting/console_ui.py:596
↓ 8 callersFunctiontrajectory_checker
Extract tool calls from an ATIF trajectory and score them with one implementation.
tests/tier3/test_error_recovery.py:919
↓ 8 callersFunctionunit_vector_similarity
Return the cosine similarity of two same-width normalized vectors. Pairwise scans normalize each vector once with :func:`normalize_embedding_vect
src/skillevaluator/embedding/client.py:81
↓ 8 callersMethodvalidate
Validate rule(s) at path for schema and structure compliance.
src/skillevaluator/validators/rules_schema.py:49
↓ 8 callersMethodvalidate
Run code risk analysis on skill(s) at path.
src/skillevaluator/validators/code_risk.py:115
↓ 8 callersMethodvalidate
Lint scripts in a skill directory or folder of skills.
src/skillevaluator/validators/script_lint.py:109
↓ 8 callersMethodvalidate
(self, skill_path: Path)
src/skillevaluator/validators/rubric_eval.py:635
↓ 8 callersFunctionvalidate_kebab_case_name
Validate that a name follows kebab-case convention. Centralizes the naming validation logic used across validators. Args: name: The
src/skillevaluator/validators/naming_utils.py:26
↓ 8 callersFunctionwrite_suite
(directory, suite)
tests/tier3/test_grader_validation.py:30
↓ 8 callersFunctionwrite_tier
(reports, tier, env, status, *, required=True)
tests/ci/test_run_tiers.py:335
↓ 7 callersFunctionEn
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 7 callersFunctionNe
(t,e,i,o,a,r={})
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 7 callersFunction_absolute_lexical
(path: Path)
src/skillevaluator/publication_identity.py:85
← previousnext →1,101–1,200 of 24,741, ranked by callers