Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/NVIDIA/SkillEvaluator
/ functions
Functions
24,741 in github.com/NVIDIA/SkillEvaluator
⨍
Functions
24,741
◇
Types & classes
1,292
↳
Endpoints
703
Function
test_scaffold_preserves_existing_dataset_tasks
(tmp_path: Path)
tests/test_harbor_scaffold_transactions.py:339
Function
test_scaffold_rejects_contract_invalid_existing_dataset_before_publish
( tmp_path: Path, entry: dict[str, object], )
tests/test_harbor_scaffold_transactions.py:172
Function
test_scaffold_rejects_escaping_case_id_before_mutation
(tmp_path: Path, case_id: str)
tests/test_harbor_scaffold_transactions.py:51
Function
test_scaffold_rejects_harbor_root_reserved_name_before_mutation
(tmp_path: Path, case_id: str)
tests/test_harbor_scaffold_transactions.py:61
Function
test_scaffold_rejects_malformed_existing_authored_state_without_with_config
( tmp_path: Path, invalid_name: str, invalid_content: str, )
tests/test_harbor_scaffold_transactions.py:108
Function
test_scaffold_rejects_symlinked_evals_root_without_touching_target
(tmp_path: Path)
tests/test_harbor_scaffold_transactions.py:70
Function
test_scaffold_starter_case_grades_the_outcome
(tmp_path: Path, command)
tests/tier3/test_dataset_baseline_fairness.py:1584
Function
test_scaffold_without_with_config_keeps_opt_in_default
(tmp_path: Path)
tests/test_harbor_scaffold_transactions.py:330
Method
test_scalar_budget_lets_1024_dimensional_embeddings_reach_the_chunk_ceiling
(self)
tests/deduplication/intra_skill/test_intra_skill_validator.py:72
Method
test_scalar_comparison_work_limit_fails_before_comparing
(self, monkeypatch)
tests/embedding/test_registry.py:512
Method
test_scalar_work_limit_is_incomplete_before_clustering
( self, mock_embed, mock_build_clusters, tmp_path: Path, monkeypatch,
tests/deduplication/intra_skill/test_intra_skill_validator.py:292
Function
test_scan_evidence_upload_holds_exactly_what_the_publisher_reads
(tmp_path)
tests/ci/test_publish_reports.py:762
Function
test_scan_excluded_name_matches_only_spellings_the_filesystem_aliases
(tmp_path)
tests/test_plugin_member_paths.py:86
Function
test_scan_gap_detection_reports_certified_paths_that_tier1_skips
(tmp_path: Path)
tests/test_publication_evidence.py:425
Function
test_scan_only_does_not_request_identity_or_generate_publisher
(repository, release_file)
tests/test_ci_onboarding.py:215
Method
test_scan_outside_a_validated_target_stays_incomplete
A direct per-directory scan, such as one partition of a larger tree, does not know what was left out. The 2.12.0 report is outside the refere
tests/validators/test_security.py:10616
Method
test_scan_outside_a_validated_target_uses_the_configured_reference_root
A direct scan accepts missing references only when the configured reference root is the scanned root.
tests/validators/test_security.py:10648
Function
test_scan_refreshes_inventory_after_context_work
(modules, tmp_path, monkeypatch)
tests/deduplication/plugin/test_private_dedup.py:264
Method
test_scan_root_entry_never_identifies_a_target
(self, tmp_path: Path, write_skill)
tests/embedding/test_catalog.py:864
Method
test_scan_root_itself_is_not_a_file_location
(self, tmp_path: Path)
tests/validators/test_skillspector_scope.py:226
Function
test_scan_shell_preserves_container_failures
(tmp_path, mode, exit_code)
tests/ci/test_evaluate_workflow.py:81
Method
test_scan_stays_incomplete_and_failed
The diagnostics change no gate: the partial report keeps the scan INCOMPLETE.
tests/validators/test_skillspector_reference_diagnostics.py:200
Function
test_scan_timeout_also_ends_the_scanner_tools
(tmp_path, monkeypatch)
tests/ci/test_run_tier1.py:1545
Function
test_scan_views_preserve_permissions_without_privilege_bits
(tmp_path, monkeypatch, source_mode, expected_mode)
tests/validators/test_private_plugin_composition.py:342
Function
test_scan_worker_input_defaults_to_four_parallel_scans
()
tests/ci/test_evaluate_workflow.py:232
Function
test_scanner_constant_folds_adjacent_python_string_literals
()
tests/test_oss_boundary.py:690
Function
test_scanner_constant_folds_arbitrary_python_string_concatenation
()
tests/test_oss_boundary.py:679
Method
test_scanner_executable_flag_makes_any_file_agent_executable
(self)
tests/validators/test_skillspector_scope.py:205
Function
test_scanner_fails_closed_for_malformed_or_unsupported_bash_ansi_c
(source: str)
tests/test_oss_boundary.py:841
Function
test_scanner_fails_closed_when_bash_ansi_c_decode_limit_is_exceeded
(monkeypatch)
tests/test_oss_boundary.py:851
Function
test_scanner_fails_closed_when_static_reconstruction_limit_is_exceeded
( monkeypatch, limit_name: str, limit: int, )
tests/test_oss_boundary.py:894
Function
test_scanner_locks_cover_every_requirement_with_hashes
()
tests/ci/test_release_image.py:139
Function
test_scanner_partitions_own_each_file_at_the_deepest_skill_once
(tmp_path, monkeypatch)
tests/validators/test_private_plugin_composition.py:195
Function
test_scanner_reconstructs_bash_dollar_prefixed_static_quotes
(quote: str)
tests/test_oss_boundary.py:747
Function
test_scanner_reconstructs_safe_constant_python_string_forms
(source: str)
tests/test_oss_boundary.py:713
Function
test_scanner_reconstructs_shell_adjacent_quotes
(path: str, source: str, expected_rule: str)
tests/test_oss_boundary.py:738
Method
test_scanner_reports_nothing_blocking_for_hooks_that_send_their_input_away
SkillSpector gives the network hooks the same findings as the plain ones.
tests/validators/test_skillspector_hook_surface.py:449
Function
test_scanner_shell_normalizes_reconstructed_python_constants
(source: str, expected_rule: str)
tests/test_oss_boundary.py:879
Function
test_scanner_source_does_not_match_its_constructed_deny_expressions
()
tests/test_oss_boundary.py:88
Function
test_scanners_row_keeps_the_version_when_only_the_llm_stage_failed
()
tests/reporting/test_tier3_validate_cases.py:534
Method
test_scans_copy_without_artifact_dirs
(self, mock_run, skill_with_artifacts)
tests/validators/test_artifact_dir_exclusion.py:133
Function
test_schema_and_harness_agree_on_where_a_nested_manifest_counts
(tmp_path: Path, folder: str, reported: bool)
tests/harness/test_overlay.py:1264
Function
test_schema_builders_and_downgrade_warning_log
Verify _build_openai_response_format, _build_anthropic_output_config, and warning log on 400 downgrade.
tests/test_llm_judge.py:950
Method
test_schema_downgrade_keeps_anthropic_effort
(self, monkeypatch: pytest.MonkeyPatch)
tests/inference/test_llm_client.py:1165
Function
test_schema_downgrade_memoized_only_on_confirmed_success
Verify target is only memoized into _SCHEMA_UNSUPPORTED_TARGETS when fallback succeeds.
tests/test_llm_judge.py:1096
Function
test_schema_error_cap_also_bounds_required
()
tests/tier3/test_f90_mcp_arguments.py:108
Function
test_schema_rejection_grammar_accepts_direct_option_errors
Verify structured schema rejection regex matches direct option error messages.
tests/test_llm_judge.py:1241
Method
test_schema_validator_mixed_valid_invalid
Test SchemaValidator with folder containing both valid and invalid skills.
tests/validators/test_folder_validation.py:70
Method
test_schema_validator_multiple_skills
Test SchemaValidator with folder containing multiple skills.
tests/validators/test_folder_validation.py:14
Function
test_scope_cannot_hide_other_tier_results
(reporter)
tests/reporting/test_tier3_scoped_reports.py:168
Function
test_scope_does_not_accept_partial_dataset_coverage
()
tests/reporting/test_tier3_scoped_reports.py:175
Function
test_scope_projection_distinguishes_clean_zero_from_skipped
()
tests/reporting/test_evidence_views.py:25
Function
test_scope_roundtrip_preserves_publication_verdict
(options, status)
tests/reporting/test_retained_publication_scope.py:52
Function
test_scoped_actions_never_inherit_another_agent_or_case_legacy_trace
()
tests/reporting/test_insights_view.py:331
Function
test_scoped_argv_lists_with_shell_characters_stay_productive
(module, argv)
tests/tier3/test_root_search_efficiency.py:250
Function
test_scoped_bash_grants_pass
(tmp_path: Path, rule: str)
tests/validators/test_plugin_static_risk_review_fixes.py:664
Function
test_scoped_native_searches_stay_productive
(module, call)
tests/tier3/test_tool_productivity.py:375
Function
test_scoped_or_requested_rules_follow_native_and_wrapper_views
(tmp_path, frontmatter)
tests/test_plugin_context_cost.py:69
Function
test_scoped_quoted_heredoc_and_redirect_targets_keep_old_credit
(module, command)
tests/tier3/test_root_search_efficiency.py:174
Function
test_scoped_reports_show_completed_agent_and_signed_lift
(baseline, lift)
tests/reporting/test_tier3_scoped_reports.py:59
Function
test_scoped_searches_patterns_and_ordinary_calls_stay_productive
(module, call)
tests/tier3/test_root_search_efficiency.py:66
Function
test_scoped_searches_stay_productive
(module, command)
tests/tier3/test_tool_productivity.py:394
Function
test_scoped_skill_allowed_tools_pass
(tmp_path: Path)
tests/validators/test_plugin_static_risk_review_fixes.py:139
Function
test_score_definition_identifies_the_versioned_dimension_policy
()
tests/golden/test_tier3_schema.py:52
Function
test_score_delta_preserves_canonical_judge_aware_lift
(lift: float | None)
tests/reporting/test_dashboard_data.py:80
Function
test_score_policy_identifiers_match_the_source
()
tests/test_metrics_reference_docs.py:231
Function
test_score_policy_metadata_is_not_a_custom_metric
()
tests/test_harbor_metrics_truth.py:309
Function
test_score_policy_mismatch_is_unavailable_not_missing_baseline
()
tests/reporting/test_tier3_unavailable_lift_labels.py:92
Function
test_score_rendering_is_isolated_from_the_ambient_decimal_context
()
tests/golden/test_benchmark_card.py:1445
Function
test_score_transition_derives_uplift_from_displayed_endpoints
( baseline: float, score: float, expected: str, )
tests/golden/test_benchmark_card.py:1345
Function
test_score_transition_does_not_publish_invalid_raw_scores
(baseline: object, score: object)
tests/golden/test_benchmark_card.py:1387
Function
test_scorecard_distinguishes_missing_quality_from_a_recorded_zero
(score)
tests/reporting/test_native_feature_restoration.py:279
Function
test_scored_and_unscored_portable_name_collision_persists_distinct_trials
(tmp_path: Path)
tests/test_issue55_collector_truth.py:2887
Function
test_scored_job_validation_still_rejects_harbor_0132_unscored_result
(tmp_path: Path)
tests/test_harbor_runtime_preflight.py:500
Function
test_scored_trial_records_judge_contract_and_every_attempt
(tmp_path: Path)
tests/tier3/test_judge_contracts.py:790
Method
test_scores_and_lift_present
(self, evaluators: dict)
tests/evaluation/test_judges.py:49
Function
test_scores_scope_and_json_only_publish
(tmp_path)
tests/ci/test_report_views.py:106
Method
test_script_based
(self, tmp_path)
tests/validators/test_quality_score.py:214
Function
test_script_double_escaped_content_does_not_produce_links
()
tests/validators/test_markdown.py:221
Function
test_script_end_scanner_uses_ascii_case_insensitive_matching
(content: str)
tests/validators/test_markdown.py:243
Function
test_script_escaped_dash_dash_greater_than_returns_to_data
(escape_boundary: str)
tests/validators/test_markdown.py:228
Function
test_script_extraction_preserves_unsafe_nonscript_attributes_and_original_markup
()
tests/reporting/test_report_markup_assertions.py:11
Function
test_script_invoked_checks_command_trajectory_only
()
tests/tier3/test_declarative_checks.py:36
Method
test_script_markers_come_from_config
(self, tmp_path: Path)
tests/validators/test_quality_score.py:1609
Function
test_script_names_in_usage_text_are_not_followed
(tmp_path: Path)
tests/validators/test_plugin_static_risk_review_fixes.py:424
Function
test_script_reference_cycle_is_read_once
(tmp_path: Path)
tests/validators/test_plugin_component_risk.py:1067
Function
test_script_run_by_a_second_level_script_is_followed
Regression: a script that a second-level hook script runs was never read, and nothing was flagged.
tests/validators/test_plugin_component_risk.py:1041
Function
test_script_source_in_the_observation_does_not_rescue_a_read
The output of ``cat run.py`` is the script's own text, not evidence.
tests/tier3/test_script_invocation_evidence.py:96
Function
test_scripts_named_through_a_root_variable_are_analyzed
(tmp_path: Path)
tests/validators/test_plugin_static_risk_review_fixes.py:504
Function
test_scripts_only_mentioned_by_root_or_dirname_are_not_followed
(tmp_path: Path, check: str)
tests/validators/test_plugin_static_risk_review_fixes.py:968
Function
test_scripts_past_the_read_cap_are_flagged
(tmp_path: Path)
tests/validators/test_plugin_hook_risk_bounds.py:144
Method
test_scripts_points_zero_skips_the_finding
(self, tmp_path: Path)
tests/validators/test_quality_score.py:1592
Function
test_scripts_run_by_root_or_dirname_are_still_followed
(tmp_path: Path, check: str)
tests/validators/test_plugin_static_risk_review_fixes.py:986
Function
test_scripts_share_one_url_scheme_check
(script: str)
tests/test_reference_api_caller.py:123
Function
test_scripts_that_are_all_entry_points_keep_listing_order
(tmp_path: Path)
tests/tier3/test_dataset_primary_script.py:126
Function
test_sdist_allowlist_only_normalizes_the_expected_distribution_root
(tmp_path: Path)
tests/test_oss_boundary.py:1280
Function
test_sdist_negative_test_uses_exact_source_allowlist_path
(tmp_path: Path)
tests/test_oss_boundary.py:999
Function
test_sdist_rejects_duplicate_members_before_associating_anchors
(tmp_path: Path)
tests/test_oss_boundary.py:1217
Function
test_sdist_rejects_links_before_allowlisting
(tmp_path: Path, member_type: bytes)
tests/test_oss_boundary.py:1233
Function
test_sdist_scan_enforces_aggregate_decompressed_byte_limit
(tmp_path: Path, monkeypatch)
tests/test_oss_boundary.py:1336
← previous
next →
21,901–22,000 of 24,741, ranked by callers