MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

Functiontest_scaffold_preserves_existing_dataset_tasks
(tmp_path: Path)
tests/test_harbor_scaffold_transactions.py:339
Functiontest_scaffold_rejects_contract_invalid_existing_dataset_before_publish
( tmp_path: Path, entry: dict[str, object], )
tests/test_harbor_scaffold_transactions.py:172
Functiontest_scaffold_rejects_escaping_case_id_before_mutation
(tmp_path: Path, case_id: str)
tests/test_harbor_scaffold_transactions.py:51
Functiontest_scaffold_rejects_harbor_root_reserved_name_before_mutation
(tmp_path: Path, case_id: str)
tests/test_harbor_scaffold_transactions.py:61
Functiontest_scaffold_rejects_malformed_existing_authored_state_without_with_config
( tmp_path: Path, invalid_name: str, invalid_content: str, )
tests/test_harbor_scaffold_transactions.py:108
Functiontest_scaffold_rejects_symlinked_evals_root_without_touching_target
(tmp_path: Path)
tests/test_harbor_scaffold_transactions.py:70
Functiontest_scaffold_starter_case_grades_the_outcome
(tmp_path: Path, command)
tests/tier3/test_dataset_baseline_fairness.py:1584
Functiontest_scaffold_without_with_config_keeps_opt_in_default
(tmp_path: Path)
tests/test_harbor_scaffold_transactions.py:330
Methodtest_scalar_budget_lets_1024_dimensional_embeddings_reach_the_chunk_ceiling
(self)
tests/deduplication/intra_skill/test_intra_skill_validator.py:72
Methodtest_scalar_comparison_work_limit_fails_before_comparing
(self, monkeypatch)
tests/embedding/test_registry.py:512
Methodtest_scalar_work_limit_is_incomplete_before_clustering
( self, mock_embed, mock_build_clusters, tmp_path: Path, monkeypatch,
tests/deduplication/intra_skill/test_intra_skill_validator.py:292
Functiontest_scan_evidence_upload_holds_exactly_what_the_publisher_reads
(tmp_path)
tests/ci/test_publish_reports.py:762
Functiontest_scan_excluded_name_matches_only_spellings_the_filesystem_aliases
(tmp_path)
tests/test_plugin_member_paths.py:86
Functiontest_scan_gap_detection_reports_certified_paths_that_tier1_skips
(tmp_path: Path)
tests/test_publication_evidence.py:425
Functiontest_scan_only_does_not_request_identity_or_generate_publisher
(repository, release_file)
tests/test_ci_onboarding.py:215
Methodtest_scan_outside_a_validated_target_stays_incomplete
A direct per-directory scan, such as one partition of a larger tree, does not know what was left out. The 2.12.0 report is outside the refere
tests/validators/test_security.py:10616
Methodtest_scan_outside_a_validated_target_uses_the_configured_reference_root
A direct scan accepts missing references only when the configured reference root is the scanned root.
tests/validators/test_security.py:10648
Functiontest_scan_refreshes_inventory_after_context_work
(modules, tmp_path, monkeypatch)
tests/deduplication/plugin/test_private_dedup.py:264
Methodtest_scan_root_entry_never_identifies_a_target
(self, tmp_path: Path, write_skill)
tests/embedding/test_catalog.py:864
Methodtest_scan_root_itself_is_not_a_file_location
(self, tmp_path: Path)
tests/validators/test_skillspector_scope.py:226
Functiontest_scan_shell_preserves_container_failures
(tmp_path, mode, exit_code)
tests/ci/test_evaluate_workflow.py:81
Methodtest_scan_stays_incomplete_and_failed
The diagnostics change no gate: the partial report keeps the scan INCOMPLETE.
tests/validators/test_skillspector_reference_diagnostics.py:200
Functiontest_scan_timeout_also_ends_the_scanner_tools
(tmp_path, monkeypatch)
tests/ci/test_run_tier1.py:1545
Functiontest_scan_views_preserve_permissions_without_privilege_bits
(tmp_path, monkeypatch, source_mode, expected_mode)
tests/validators/test_private_plugin_composition.py:342
Functiontest_scan_worker_input_defaults_to_four_parallel_scans
()
tests/ci/test_evaluate_workflow.py:232
Functiontest_scanner_constant_folds_adjacent_python_string_literals
()
tests/test_oss_boundary.py:690
Functiontest_scanner_constant_folds_arbitrary_python_string_concatenation
()
tests/test_oss_boundary.py:679
Methodtest_scanner_executable_flag_makes_any_file_agent_executable
(self)
tests/validators/test_skillspector_scope.py:205
Functiontest_scanner_fails_closed_for_malformed_or_unsupported_bash_ansi_c
(source: str)
tests/test_oss_boundary.py:841
Functiontest_scanner_fails_closed_when_bash_ansi_c_decode_limit_is_exceeded
(monkeypatch)
tests/test_oss_boundary.py:851
Functiontest_scanner_fails_closed_when_static_reconstruction_limit_is_exceeded
( monkeypatch, limit_name: str, limit: int, )
tests/test_oss_boundary.py:894
Functiontest_scanner_locks_cover_every_requirement_with_hashes
()
tests/ci/test_release_image.py:139
Functiontest_scanner_partitions_own_each_file_at_the_deepest_skill_once
(tmp_path, monkeypatch)
tests/validators/test_private_plugin_composition.py:195
Functiontest_scanner_reconstructs_bash_dollar_prefixed_static_quotes
(quote: str)
tests/test_oss_boundary.py:747
Functiontest_scanner_reconstructs_safe_constant_python_string_forms
(source: str)
tests/test_oss_boundary.py:713
Functiontest_scanner_reconstructs_shell_adjacent_quotes
(path: str, source: str, expected_rule: str)
tests/test_oss_boundary.py:738
Methodtest_scanner_reports_nothing_blocking_for_hooks_that_send_their_input_away
SkillSpector gives the network hooks the same findings as the plain ones.
tests/validators/test_skillspector_hook_surface.py:449
Functiontest_scanner_shell_normalizes_reconstructed_python_constants
(source: str, expected_rule: str)
tests/test_oss_boundary.py:879
Functiontest_scanner_source_does_not_match_its_constructed_deny_expressions
()
tests/test_oss_boundary.py:88
Functiontest_scanners_row_keeps_the_version_when_only_the_llm_stage_failed
()
tests/reporting/test_tier3_validate_cases.py:534
Methodtest_scans_copy_without_artifact_dirs
(self, mock_run, skill_with_artifacts)
tests/validators/test_artifact_dir_exclusion.py:133
Functiontest_schema_and_harness_agree_on_where_a_nested_manifest_counts
(tmp_path: Path, folder: str, reported: bool)
tests/harness/test_overlay.py:1264
Functiontest_schema_builders_and_downgrade_warning_log
Verify _build_openai_response_format, _build_anthropic_output_config, and warning log on 400 downgrade.
tests/test_llm_judge.py:950
Methodtest_schema_downgrade_keeps_anthropic_effort
(self, monkeypatch: pytest.MonkeyPatch)
tests/inference/test_llm_client.py:1165
Functiontest_schema_downgrade_memoized_only_on_confirmed_success
Verify target is only memoized into _SCHEMA_UNSUPPORTED_TARGETS when fallback succeeds.
tests/test_llm_judge.py:1096
Functiontest_schema_error_cap_also_bounds_required
()
tests/tier3/test_f90_mcp_arguments.py:108
Functiontest_schema_rejection_grammar_accepts_direct_option_errors
Verify structured schema rejection regex matches direct option error messages.
tests/test_llm_judge.py:1241
Methodtest_schema_validator_mixed_valid_invalid
Test SchemaValidator with folder containing both valid and invalid skills.
tests/validators/test_folder_validation.py:70
Methodtest_schema_validator_multiple_skills
Test SchemaValidator with folder containing multiple skills.
tests/validators/test_folder_validation.py:14
Functiontest_scope_cannot_hide_other_tier_results
(reporter)
tests/reporting/test_tier3_scoped_reports.py:168
Functiontest_scope_does_not_accept_partial_dataset_coverage
()
tests/reporting/test_tier3_scoped_reports.py:175
Functiontest_scope_projection_distinguishes_clean_zero_from_skipped
()
tests/reporting/test_evidence_views.py:25
Functiontest_scope_roundtrip_preserves_publication_verdict
(options, status)
tests/reporting/test_retained_publication_scope.py:52
Functiontest_scoped_actions_never_inherit_another_agent_or_case_legacy_trace
()
tests/reporting/test_insights_view.py:331
Functiontest_scoped_argv_lists_with_shell_characters_stay_productive
(module, argv)
tests/tier3/test_root_search_efficiency.py:250
Functiontest_scoped_bash_grants_pass
(tmp_path: Path, rule: str)
tests/validators/test_plugin_static_risk_review_fixes.py:664
Functiontest_scoped_native_searches_stay_productive
(module, call)
tests/tier3/test_tool_productivity.py:375
Functiontest_scoped_or_requested_rules_follow_native_and_wrapper_views
(tmp_path, frontmatter)
tests/test_plugin_context_cost.py:69
Functiontest_scoped_quoted_heredoc_and_redirect_targets_keep_old_credit
(module, command)
tests/tier3/test_root_search_efficiency.py:174
Functiontest_scoped_reports_show_completed_agent_and_signed_lift
(baseline, lift)
tests/reporting/test_tier3_scoped_reports.py:59
Functiontest_scoped_searches_patterns_and_ordinary_calls_stay_productive
(module, call)
tests/tier3/test_root_search_efficiency.py:66
Functiontest_scoped_searches_stay_productive
(module, command)
tests/tier3/test_tool_productivity.py:394
Functiontest_scoped_skill_allowed_tools_pass
(tmp_path: Path)
tests/validators/test_plugin_static_risk_review_fixes.py:139
Functiontest_score_definition_identifies_the_versioned_dimension_policy
()
tests/golden/test_tier3_schema.py:52
Functiontest_score_delta_preserves_canonical_judge_aware_lift
(lift: float | None)
tests/reporting/test_dashboard_data.py:80
Functiontest_score_policy_identifiers_match_the_source
()
tests/test_metrics_reference_docs.py:231
Functiontest_score_policy_metadata_is_not_a_custom_metric
()
tests/test_harbor_metrics_truth.py:309
Functiontest_score_policy_mismatch_is_unavailable_not_missing_baseline
()
tests/reporting/test_tier3_unavailable_lift_labels.py:92
Functiontest_score_rendering_is_isolated_from_the_ambient_decimal_context
()
tests/golden/test_benchmark_card.py:1445
Functiontest_score_transition_derives_uplift_from_displayed_endpoints
( baseline: float, score: float, expected: str, )
tests/golden/test_benchmark_card.py:1345
Functiontest_score_transition_does_not_publish_invalid_raw_scores
(baseline: object, score: object)
tests/golden/test_benchmark_card.py:1387
Functiontest_scorecard_distinguishes_missing_quality_from_a_recorded_zero
(score)
tests/reporting/test_native_feature_restoration.py:279
Functiontest_scored_and_unscored_portable_name_collision_persists_distinct_trials
(tmp_path: Path)
tests/test_issue55_collector_truth.py:2887
Functiontest_scored_job_validation_still_rejects_harbor_0132_unscored_result
(tmp_path: Path)
tests/test_harbor_runtime_preflight.py:500
Functiontest_scored_trial_records_judge_contract_and_every_attempt
(tmp_path: Path)
tests/tier3/test_judge_contracts.py:790
Methodtest_scores_and_lift_present
(self, evaluators: dict)
tests/evaluation/test_judges.py:49
Functiontest_scores_scope_and_json_only_publish
(tmp_path)
tests/ci/test_report_views.py:106
Methodtest_script_based
(self, tmp_path)
tests/validators/test_quality_score.py:214
Functiontest_script_double_escaped_content_does_not_produce_links
()
tests/validators/test_markdown.py:221
Functiontest_script_end_scanner_uses_ascii_case_insensitive_matching
(content: str)
tests/validators/test_markdown.py:243
Functiontest_script_escaped_dash_dash_greater_than_returns_to_data
(escape_boundary: str)
tests/validators/test_markdown.py:228
Functiontest_script_extraction_preserves_unsafe_nonscript_attributes_and_original_markup
()
tests/reporting/test_report_markup_assertions.py:11
Functiontest_script_invoked_checks_command_trajectory_only
()
tests/tier3/test_declarative_checks.py:36
Methodtest_script_markers_come_from_config
(self, tmp_path: Path)
tests/validators/test_quality_score.py:1609
Functiontest_script_names_in_usage_text_are_not_followed
(tmp_path: Path)
tests/validators/test_plugin_static_risk_review_fixes.py:424
Functiontest_script_reference_cycle_is_read_once
(tmp_path: Path)
tests/validators/test_plugin_component_risk.py:1067
Functiontest_script_run_by_a_second_level_script_is_followed
Regression: a script that a second-level hook script runs was never read, and nothing was flagged.
tests/validators/test_plugin_component_risk.py:1041
Functiontest_script_source_in_the_observation_does_not_rescue_a_read
The output of ``cat run.py`` is the script's own text, not evidence.
tests/tier3/test_script_invocation_evidence.py:96
Functiontest_scripts_named_through_a_root_variable_are_analyzed
(tmp_path: Path)
tests/validators/test_plugin_static_risk_review_fixes.py:504
Functiontest_scripts_only_mentioned_by_root_or_dirname_are_not_followed
(tmp_path: Path, check: str)
tests/validators/test_plugin_static_risk_review_fixes.py:968
Functiontest_scripts_past_the_read_cap_are_flagged
(tmp_path: Path)
tests/validators/test_plugin_hook_risk_bounds.py:144
Methodtest_scripts_points_zero_skips_the_finding
(self, tmp_path: Path)
tests/validators/test_quality_score.py:1592
Functiontest_scripts_run_by_root_or_dirname_are_still_followed
(tmp_path: Path, check: str)
tests/validators/test_plugin_static_risk_review_fixes.py:986
Functiontest_scripts_share_one_url_scheme_check
(script: str)
tests/test_reference_api_caller.py:123
Functiontest_scripts_that_are_all_entry_points_keep_listing_order
(tmp_path: Path)
tests/tier3/test_dataset_primary_script.py:126
Functiontest_sdist_allowlist_only_normalizes_the_expected_distribution_root
(tmp_path: Path)
tests/test_oss_boundary.py:1280
Functiontest_sdist_negative_test_uses_exact_source_allowlist_path
(tmp_path: Path)
tests/test_oss_boundary.py:999
Functiontest_sdist_rejects_duplicate_members_before_associating_anchors
(tmp_path: Path)
tests/test_oss_boundary.py:1217
Functiontest_sdist_rejects_links_before_allowlisting
(tmp_path: Path, member_type: bytes)
tests/test_oss_boundary.py:1233
Functiontest_sdist_scan_enforces_aggregate_decompressed_byte_limit
(tmp_path: Path, monkeypatch)
tests/test_oss_boundary.py:1336
← previousnext →21,901–22,000 of 24,741, ranked by callers