MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

Functiontest_skill_frontmatter_hooks_are_analyzed
(tmp_path: Path)
tests/validators/test_plugin_static_risk_review_fixes.py:104
Functiontest_skill_listing_body_and_mcp_unknown_schema_are_separate
(tmp_path)
tests/test_plugin_context_cost.py:34
Functiontest_skill_manifest_file_keeps_default_reports_outside_the_skill
( monkeypatch: pytest.MonkeyPatch, tmp_path: Path )
tests/cli/test_validate_manifest_file.py:245
Functiontest_skill_manifest_file_refuses_an_output_dir_inside_the_skill
( monkeypatch: pytest.MonkeyPatch, tmp_path: Path )
tests/cli/test_validate_manifest_file.py:261
Functiontest_skill_manifest_file_scans_the_whole_skill
(tmp_path: Path)
tests/cli/test_validate_manifest_file.py:69
Functiontest_skill_manifest_file_validates_like_its_folder
(tmp_path: Path)
tests/cli/test_validate_manifest_file.py:41
Functiontest_skill_manifest_linked_to_a_file_with_another_name_is_refused_without_no_dedup
( monkeypatch: pytest.MonkeyPatch, tmp_path: Path )
tests/cli/test_validate_manifest_file.py:169
Functiontest_skill_manifest_linked_to_a_file_with_another_name_scans_the_whole_skill
( tmp_path: Path, manifest_name: str, type_args: tuple[str, ...] )
tests/cli/test_validate_manifest_file.py:148
Methodtest_skill_mcp_subagent_and_command_keep_identity_order_and_outcome
(self)
tests/tier3/test_plugin_signals.py:106
Methodtest_skill_md_case_insensitive
Test that skill.md (lowercase) is found but produces a naming warning.
tests/validators/test_schema.py:121
Functiontest_skill_md_in_the_skill_itself_still_warns
(assertion)
tests/tier3/test_dataset_baseline_fairness.py:720
Functiontest_skill_md_that_is_not_utf8_stops_the_catalog_with_a_frontmatter_reason
( consumer, tmp_path, monkeypatch, encoding )
tests/ci/test_run_tiers.py:1071
Methodtest_skill_md_uppercase_no_naming_warning
Test that SKILL.md (uppercase) does NOT produce a naming warning.
tests/validators/test_schema.py:151
Functiontest_skill_metrics_apply_to_skill_and_matching_acceptable_alternate
()
tests/tier3/test_baseline_dimension_applicability.py:34
Functiontest_skill_modules_and_package_aliases_are_classified
(checker)
tests/tier3/test_error_recovery.py:159
Functiontest_skill_name_is_read_from_any_yaml_spelling_of_the_name
A missing command named after the skill is the agent's choice however the manifest spells the name.
tests/tier3/test_error_recovery.py:329
Functiontest_skill_name_matches_yaml_safe_load
The verifier's reader agrees with the parser the schema validator uses.
tests/tier3/test_error_recovery.py:343
Functiontest_skill_only_catalog_built_from_a_declared_root_is_self
(tmp_path: Path, root: str, relative: str)
tests/deduplication/plugin/test_openclaw_profile.py:169
Functiontest_skill_outside_a_skills_folder_has_no_catalog_rules
(tmp_path: Path)
tests/harness/test_overlay.py:368
Functiontest_skill_path_and_non_skill_file
(checker)
tests/tier3/test_error_recovery.py:93
Functiontest_skill_path_as_tool_argument_does_not_mean_skill_uses_tool
(checker)
tests/tier3/test_error_recovery.py:151
Functiontest_skill_root_accepts_windows_style_separators
(tmp_path: Path)
tests/test_precommit.py:156
Functiontest_skill_root_finds_lowercase_manifest
(tmp_path: Path)
tests/test_precommit.py:522
Functiontest_skill_root_finds_repo_root_skill_from_relative_path
(tmp_path: Path)
tests/test_precommit.py:344
Functiontest_skill_root_ignores_excluded_dir_names_above_cwd
(tmp_path: Path)
tests/test_precommit.py:113
Functiontest_skill_root_ignores_files_outside_a_skill
(tmp_path: Path)
tests/test_precommit.py:70
Functiontest_skill_root_ignores_manifest_above_cwd
(tmp_path: Path)
tests/test_precommit.py:353
Functiontest_skill_root_keeps_literal_backslash_in_posix_path
(tmp_path: Path, monkeypatch: pytest.MonkeyPatch)
tests/test_precommit.py:168
Functiontest_skill_root_maps_nested_skill_not_outer
(tmp_path: Path)
tests/test_precommit.py:58
Functiontest_skill_root_matches_manifest_names_exactly
(tmp_path: Path)
tests/test_precommit.py:531
Functiontest_skill_root_skips_deleted_skill
(tmp_path: Path)
tests/test_precommit.py:83
Functiontest_skill_root_skips_eval_fixture_manifest
(tmp_path: Path)
tests/test_precommit.py:102
Functiontest_skill_root_skips_version_snapshot_manifest
(tmp_path: Path)
tests/test_precommit.py:93
Functiontest_skill_runtime_env_cannot_redirect_runta_or_mosaic_credentials
(env_mode: str, name: str)
tests/test_harbor_environment_contract.py:392
Functiontest_skill_runtime_env_cannot_reenable_background_tasks
(tmp_path: Path)
tests/test_harbor_claude_background_tasks.py:57
Functiontest_skill_runtime_env_cannot_reenable_refusal_fallback
(tmp_path: Path)
tests/test_harbor_claude_refusal_fallback.py:457
Functiontest_skill_runtime_env_cannot_set_a_judge_policy
(name)
tests/tier3/test_judge_policies.py:860
Methodtest_skill_scanned_on_its_own_inside_a_folder_stays_incomplete
``security-scan`` on a folder scans each skill in it on its own; a line may name a file beside it.
tests/validators/test_security.py:10528
Methodtest_skill_scanned_on_its_own_inside_a_folder_stays_incomplete
(self, mock_tools, tmp_path: Path)
tests/validators/test_security.py:11303
Functiontest_skill_script_checks_are_not_applicable_without_the_skill
()
tests/tier3/test_declarative_checks.py:144
Functiontest_skill_script_missing_executable_is_environment_failure
(checker, diagnostic)
tests/tier3/test_error_recovery.py:122
Functiontest_skill_tool_observation_fallback_is_preserved
A non-read tool reporting the script ran keeps the 0.75 fallback.
tests/tier3/test_script_invocation_evidence.py:180
Functiontest_skill_use_attribution_tracks_activation_before_the_sink
(security)
tests/tier3/test_private_plugin_security.py:128
Functiontest_skill_use_is_found_in_a_field_of_any_shape
(field, value)
tests/tier3/test_dataset_baseline_fairness.py:1905
Functiontest_skill_validate_selects_tiers_and_prepares_dataset
(monkeypatch, tmp_path, flags, expected)
tests/cli/test_validate_defaults.py:35
Functiontest_skill_with_python_and_node_scripts_still_needs_run_script
(tmp_path)
tests/validators/test_quality_score.py:2169
Functiontest_skill_without_a_readable_name_records_no_name
(tmp_path, frontmatter)
tests/tier3/test_error_recovery.py:372
Functiontest_skill_without_valid_frontmatter_stops_the_catalog_with_a_frontmatter_reason
( consumer, tmp_path, monkeypatch, value )
tests/ci/test_run_tiers.py:1049
Methodtest_skillevaluator_hint_does_not_reference_pypi
skillevaluator is not published to PyPI; the hint must use the git source.
tests/utils/test_tool_runner.py:449
Functiontest_skills_changed_cli_matches_the_launcher_decision
The GitLab aggregate and paid tiers recompute the skip with this exact command.
tests/ci/test_run_tier1_image.py:339
Methodtest_skills_nested_in_the_target_are_not_catalog_tree_evidence
(self, tmp_path: Path, write_skill)
tests/embedding/test_catalog.py:764
Methodtest_skillspector_10_confidence_captured
Skillspector 1.0 confidence float is captured in finding metadata.
tests/validators/test_security.py:8161
Methodtest_skillspector_10_metadata_captured
Top-level skill, components, and metadata from skillspector 1.0 are captured.
tests/validators/test_security.py:8308
Methodtest_skillspector_10_null_finding_uses_explanation_fallback
When skillspector returns null for finding field, message falls back to explanation.
tests/validators/test_security.py:8218
Methodtest_skillspector_2_10_partial_scan_reports_incomplete_before_recommendation
LOW/CAUTION is the documented projection of an incomplete 2.10 scan.
tests/validators/test_security.py:2510
Methodtest_skillspector_2_13_3_passwd_mount_keeps_llm_inspection_incomplete
( self, mock_tools, sample_skill_dir: Path, reason_code: str, )
tests/validators/test_security.py:3633
Methodtest_skillspector_accepts_captured_no_llm_report
( self, mock_tools, sample_skill_dir: Path, version: str, )
tests/validators/test_security.py:3469
Methodtest_skillspector_accepts_fully_covered_documentation_only_report
( self, mock_tools, sample_skill_dir: Path, )
tests/validators/test_security.py:3367
Methodtest_skillspector_accepts_legacy_report_without_completeness
( self, mock_tools, sample_skill_dir: Path, )
tests/validators/test_security.py:3988
Methodtest_skillspector_accepts_occurrence_expanded_finding_count
( self, mock_tools, sample_skill_dir: Path, )
tests/validators/test_security.py:4278
Methodtest_skillspector_accepts_post_filter_count_before_report_deduplication
( self, mock_tools, sample_skill_dir: Path, )
tests/validators/test_security.py:4245
Methodtest_skillspector_accepts_valid_clean_report
( self, mock_tools, sample_skill_dir: Path, )
tests/validators/test_security.py:3346
Methodtest_skillspector_accepts_valid_findings_report
(self, mock_tools, sample_skill_dir: Path)
tests/validators/test_security.py:4222
Methodtest_skillspector_advisory_issues_show_clear_message
When only medium/low issues exist, success message should be descriptive.
tests/validators/test_security.py:8408
Methodtest_skillspector_aggregate_policy_risk_fails_without_high_issue
( self, mock_tools, sample_skill_dir: Path, )
tests/validators/test_security.py:4310
Methodtest_skillspector_analyzer_partial_scan_preserves_valid_high_findings
( self, mock_tools, sample_skill_dir: Path, analyzer_state: str, )
tests/validators/test_security.py:2576
Methodtest_skillspector_argv_uses_resolved_path
(self, mock_run, malicious_skill_path)
tests/validators/test_scanner_path_resolution.py:149
Functiontest_skillspector_baseline_and_skip_both_reach_the_run
``--skip`` on another check must not drop the baseline, and vice versa.
tests/test_commands.py:120
Functiontest_skillspector_baseline_must_name_an_existing_file
(tmp_path: Path)
tests/test_commands.py:163
Methodtest_skillspector_baseline_outcome_accounts_for_each_stage
LLM meta-analysis rewrites findings, so static fingerprints can stop matching in that stage.
tests/validators/test_security.py:2428
Methodtest_skillspector_baseline_reaches_llm_enrichment_stage
( self, mock_tools, sample_skill_dir: Path, tmp_path: Path, monkeypatch )
tests/validators/test_security.py:2403
Methodtest_skillspector_baseline_rejection_exit_points_at_the_baseline
SkillSpector rejects some baselines only at load time, with exit code 2 and redacted diagnostics.
tests/validators/test_security.py:2251
Methodtest_skillspector_baseline_rejections_name_the_actual_problem
A shape, date, or expansion error is reported as such, not as an unbounded rule or a crash.
tests/validators/test_security.py:2339
Methodtest_skillspector_baseline_suppressions_do_not_loosen_risk_reconciliation
Suppressed rows are not unseen occurrences of the active finding.
tests/validators/test_security.py:1969
Methodtest_skillspector_baseline_suppressions_do_not_lower_filtered_risk_floor
Suppressed rows must not weaken the score floor kept after generated-artifact issues are removed.
tests/validators/test_security.py:2000
Methodtest_skillspector_bedrock_environment_keeps_only_aws_chain
(self, monkeypatch)
tests/validators/test_security.py:5484
Methodtest_skillspector_captured_2_13_3_passwd_mount
( self, mock_tools, sample_skill_dir: Path, case: str, provider_availa
tests/validators/test_security.py:3585
Methodtest_skillspector_captured_classification_distinct_pe3
( self, mock_tools, sample_skill_dir: Path, mutation: str | None, )
tests/validators/test_security.py:3515
Methodtest_skillspector_child_environment_explicit_compatible_config_overrides_ambient_openai
( self, monkeypatch, )
tests/validators/test_security.py:5520
Methodtest_skillspector_child_environment_maps_generic_openai_key
Verify SKILL_EVAL_LLM_PROVIDER=openai with generic key forwards OPENAI_API_KEY.
tests/validators/test_security.py:5569
Methodtest_skillspector_child_environment_maps_openai_compatible_provider
(self, monkeypatch)
tests/validators/test_security.py:5502
Methodtest_skillspector_child_environment_maps_supported_public_provider_defaults
( self, monkeypatch, provider: str, credential: str, model: str,
tests/validators/test_security.py:5462
Methodtest_skillspector_child_environment_maps_vertex_openapi_adc
Verify SkillSpector child receives OPENAI_API_KEY with ADC under both provider aliases.
tests/validators/test_security.py:5543
Methodtest_skillspector_compacted_identity_rejects_conflicting_score_fields
( self, mock_tools, sample_skill_dir: Path, conflicting_fields: dict, )
tests/validators/test_security.py:5052
Methodtest_skillspector_compacted_score_does_not_overstate_executable_evidence
( self, mock_tools, sample_skill_dir: Path, )
tests/validators/test_security.py:4972
Methodtest_skillspector_compacted_score_does_not_trust_representative_confidence
( self, mock_tools, sample_skill_dir: Path, )
tests/validators/test_security.py:5014
Methodtest_skillspector_complete_report_accepts_future_not_applicable_analyzer
( self, mock_tools, sample_skill_dir: Path, )
tests/validators/test_security.py:4942
Methodtest_skillspector_complete_report_requires_universal_analyzer_work
( self, mock_tools, sample_skill_dir: Path, skillspector_version: str,
tests/validators/test_security.py:4898
Methodtest_skillspector_complete_summary_rejects_invalid_details
( self, mock_tools, sample_skill_dir: Path, completeness_detail: dict | None,
tests/validators/test_security.py:3043
Functiontest_skillspector_defaults_cover_every_provider_and_keep_supported_claude_models
()
tests/test_provider_config.py:89
Methodtest_skillspector_documentation_only_exception_requires_exact_applicability_evidence
( self, mock_tools, sample_skill_dir: Path, invalid_evidence: str, )
tests/validators/test_security.py:3385
Methodtest_skillspector_documentation_only_exception_requires_full_coverage
( self, mock_tools, sample_skill_dir: Path, incomplete_detail: dict, )
tests/validators/test_security.py:3424
Methodtest_skillspector_executable_skill_cannot_use_documentation_only_exception
( self, mock_tools, sample_skill_dir: Path, )
tests/validators/test_security.py:3448
Methodtest_skillspector_exit_2_without_a_baseline_has_no_baseline_hint
( self, mock_tools, sample_skill_dir: Path )
tests/validators/test_security.py:2271
Methodtest_skillspector_failed_report_does_not_process_findings
( self, mock_tools, sample_skill_dir: Path, failure_mode: str, expecte
tests/validators/test_security.py:2790
Methodtest_skillspector_filtered_score_does_not_use_incomplete_serialized_bound
( self, mock_tools, sample_skill_dir: Path, )
tests/validators/test_security.py:5180
Methodtest_skillspector_filtered_score_retains_reported_risk_floor
( self, mock_tools, sample_skill_dir: Path, filtered_confidence: float,
tests/validators/test_security.py:5133
Methodtest_skillspector_filtered_score_uses_executable_component_evidence
( self, mock_tools, sample_skill_dir: Path, )
tests/validators/test_security.py:4654
Methodtest_skillspector_findings_for_generated_artifacts_are_ignored
Generated publishing artifacts should not fail Tier 1 security scanning.
tests/validators/test_security.py:6534
Methodtest_skillspector_fingerprint_version_mismatch_is_reported
( self, mock_tools, blocking: bool, status: str, sample_skill_dir: Path, tmp_path: Path )
tests/validators/test_security.py:2064
← previousnext →22,401–22,500 of 24,741, ranked by callers