MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

Functiontest_compose_stream_callback_writes_and_closes_stdin
( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, )
tests/test_harbor_secure_docker_environment.py:424
Functiontest_compose_stream_check_failure_redacts_replacement_token_once
( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, )
tests/test_harbor_secure_docker_environment.py:1285
Functiontest_compose_stream_external_cancellation_reaps_cooperative_callback_task
( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, )
tests/test_harbor_secure_docker_environment.py:1420
Functiontest_compose_stream_nested_secret_union_does_not_starve_event_loop
( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, )
tests/test_harbor_secure_docker_environment.py:975
Functiontest_compose_stream_redaction_marker_cannot_disclose_or_synthesize_secrets
( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, secrets: tuple[str, ...], raw_output: str,
tests/test_harbor_secure_docker_environment.py:1334
Functiontest_composition_supplies_both_snapshots_before_a_single_html_render
()
tests/reporting/test_plugin_evidence_report.py:131
Functiontest_compound_command_checks_later_read_commands_for_skill_md
()
tests/tier3/test_checks_path_extraction.py:657
Functiontest_compound_or_deferred_execution_keeps_conservative_link_evidence
(security_check, command)
tests/tier3/test_destructive_command_paths.py:808
Functiontest_compressed_reply_cannot_bypass_the_host_probe_body_limit
()
tests/tier3/test_private_evidence_review2.py:20
Functiontest_compute_adc_job_timeout_falls_back_to_effective_ceiling_on_invalid_override
Fall back to effective_ceiling when configured_timeout_sec is malformed, non-positive, or non-finite.
tests/test_provider_config.py:690
Methodtest_compute_dimensions_no_llm_uses_deterministic
(self, evaluators: dict)
tests/evaluation/test_judges.py:79
Functiontest_computed_allow_decision_is_auto_approve
Regression: an allow value held apart from permissionDecision (variable, helper, ternary, jq, printf) gave no auto_approve flag and no finding.
tests/validators/test_plugin_component_risk.py:888
Functiontest_conclusions_separate_unavailable_trials_from_threshold_misses
(has_threshold_miss: bool)
tests/reporting/test_unified_tier3_report.py:657
Functiontest_conclusions_without_case_grounding_do_not_receive_positional_evidence
()
tests/reporting/test_unified_tier3_report.py:645
Methodtest_concurrent_clusters_keep_their_own_answering_model
Through the real client: each worker thread reads the model that answered its own call.
tests/deduplication/intra_skill/test_intra_skill_validator.py:1563
Functiontest_concurrent_first_key_creation_publishes_one_complete_key
( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, )
tests/test_harbor_output_provenance.py:306
Functiontest_concurrent_new_file_is_not_overwritten
(repository, monkeypatch)
tests/test_ci_onboarding_writes.py:84
Functiontest_concurrent_programmatic_generation_has_no_cross_talk
(tmp_path)
tests/tier3/test_generate_dataset_results.py:474
Functiontest_concurrent_secure_handoff_scopes_are_isolated_and_reset
( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, )
tests/test_harbor_secure_docker_environment.py:6506
Functiontest_concurrent_subagents_keep_separate_windows
Harbor 0.24 interleaves concurrent subagents by time and tags each step with ``agent_id``.
tests/tier3/test_f90_signals_handoff.py:220
Functiontest_concurrent_timeout_keeps_finished_results_and_starts_no_more_skills
(tmp_path, capsys)
tests/ci/test_run_tier1.py:1402
Functiontest_condition_counts_cases_without_inferring_them_from_attempts
(rewards, options, coverage, status)
tests/tier3/test_private_case_coverage_review.py:68
Functiontest_condition_custom_metric_union_fails_before_summary_or_custom_lift
(tmp_path: Path)
tests/test_issue55_collector_truth.py:3344
Functiontest_condition_marks_incomplete_reward_as_unscored
()
tests/test_harbor_metrics_truth.py:1311
Functiontest_condition_summary_counts_judge_failures_beyond_bounded_detail_sample
()
tests/test_harbor_metrics_truth.py:923
Functiontest_condition_summary_counts_step_judge_failures_once_per_logical_attempt
()
tests/test_harbor_metrics_truth.py:943
Methodtest_confidence_bounds_are_accepted_as_floats
(self, make_cluster, confidence: object, expected: float)
tests/deduplication/intra_skill/test_intra_llm_analyzer.py:525
Methodtest_confidence_outside_zero_to_one_is_rejected
(self, make_cluster, confidence: object)
tests/deduplication/intra_skill/test_intra_llm_analyzer.py:512
Methodtest_config_allowlists_generated_dirs_but_not_authored_evals
(self)
tests/validators/test_artifact_dir_exclusion.py:406
Functiontest_config_declared_substitution_is_resolved_without_leaking_source_variable
( monkeypatch: pytest.MonkeyPatch, )
tests/test_harbor_runner_environment.py:2204
Functiontest_config_errors_accept_decoys_and_timeout_multiplier
(tmp_path: Path)
tests/test_nemo_gym_backend.py:1236
Functiontest_config_errors_let_cli_overrides_win
(tmp_path: Path)
tests/test_nemo_gym_backend.py:1361
Functiontest_config_identity_reasons
(field: str, value: object, reason: str)
tests/test_baseline_store.py:98
Functiontest_config_key_must_be_a_boolean
(tmp_path: Path)
tests/tier3/test_baseline_timeouts.py:1752
Methodtest_config_loading
Test that license config loads correctly.
tests/validators/test_license.py:300
Functiontest_config_not_object_blocked
()
tests/validators/test_mcp_static.py:492
Methodtest_config_skips_test_directories_not_substring_names
(self)
tests/validators/test_artifact_dir_exclusion.py:484
Functiontest_config_walk_reads_tables_under_documentation_key_names
Regression: the walk skipped everything under a key such as 'notes', so a server the author named 'notes' was never inspected.
tests/validators/test_plugin_permissions.py:22
Functiontest_config_walk_skips_prose_under_documentation_keys
(prose: Any)
tests/validators/test_plugin_permissions.py:32
Methodtest_configured_budget_reaches_the_collector
(self, tmp_path, monkeypatch)
tests/validators/test_rubric_eval.py:614
Functiontest_configured_credential_values_are_redacted_from_both_arms
(monkeypatch: pytest.MonkeyPatch)
tests/reporting/test_tier3_failure_reasons.py:284
Functiontest_configured_native_source_fails_when_directory_is_missing
(tmp_path: Path)
tests/tier3/test_tier3_source.py:85
Functiontest_configured_root_with_del_is_rejected
(repository)
tests/ci/test_select_changed_skills.py:84
Functiontest_configured_roots_are_literal_not_git_pathspec_globs
(repository)
tests/ci/test_select_changed_skills.py:149
Functiontest_confirmed_failure_and_incomplete_scanner_survive_retained_publication
(tmp_path)
tests/ci/test_run_tier1.py:676
Functiontest_confirmed_failure_never_gets_green_pass_in_full_cli_details
()
tests/reporting/test_final_render_context.py:244
Functiontest_confirmed_failure_precedes_incompleteness_in_retained_roundtrip
(payload, same_validator)
tests/reporting/test_retained_html.py:91
Functiontest_confirmed_failure_survives_a_folder_merge
A folder keeps a member's confirmed failure; a member's gate failure alone stays missing evidence.
tests/reporting/test_shared_run_status.py:106
Functiontest_confirmed_legacy_failure_survives_retained_roundtrip
()
tests/reporting/test_retained_html.py:111
Functiontest_confirmed_legacy_failure_survives_separate_missing_evidence
()
tests/reporting/test_shared_run_status.py:96
Functiontest_confirmed_skill_failure_precedes_another_skills_incomplete_scan
(tmp_path)
tests/ci/test_run_tier1.py:705
Functiontest_conflict_marked_skipped_tier3_cannot_waive_publication_tiers
( conflict_location: str, )
tests/reporting/test_advisory_skip_verdict.py:4091
Methodtest_conflict_propagates_out_of_the_payload_builder
(self)
tests/test_benchmark_source_provenance.py:820
Functiontest_conflicting_aliases_and_multirow_trials_fail_closed
()
tests/reporting/test_tier3_expectations.py:216
Methodtest_conflicting_allowed_licenses_fail_closed
MIT frontmatter vs Apache LICENSE must not silently pick either side.
tests/validators/test_license.py:417
Methodtest_conflicting_carriers_fail_closed
(self)
tests/test_benchmark_source_provenance.py:1443
Methodtest_conflicting_channels_fail_closed
(self)
tests/test_benchmark_source_provenance.py:813
Functiontest_conflicting_group_pins_are_each_audited
(tmp_path, monkeypatch)
tests/validators/test_target_dependencies.py:52
Functiontest_conflicting_native_source_cannot_publish
(trusted, tmp_path)
tests/ci/test_benchmark_publish.py:254
Functiontest_conflicting_native_source_is_retained_and_marked_incomplete
(tmp_path, field, value)
tests/ci/test_benchmark_reports.py:327
Functiontest_conflicting_policy_inside_one_payload_fails_closed
( key: str, top_value: bool, )
tests/reporting/test_advisory_skip_verdict.py:920
Functiontest_conflicting_publisher_does_not_partially_write_scan
(repository, release_file)
tests/test_ci_onboarding.py:224
Functiontest_conflicting_refs_with_the_same_owner_are_rejected
()
tests/tier3/test_suggestion_grounding.py:134
Functiontest_conflicting_result_level_benchmark_policy_fails_closed_independent_of_order
( reverse: bool, )
tests/golden/test_benchmark_card.py:881
Functiontest_conflicting_source_identity_is_explicit_and_never_a_static_pass
()
tests/reporting/test_private_static_reports.py:464
Functiontest_conflicting_tier3_payload_policies_fail_closed_independent_of_order
(reverse: bool)
tests/reporting/test_advisory_skip_verdict.py:894
Functiontest_confusables_table_is_pinned_to_unicode_17
()
tests/test_publication_text.py:20
Methodtest_connection_error_does_not_echo_endpoint_credentials
(self)
tests/embedding/test_client.py:191
Functiontest_consecutive_dots_inside_case_id_are_safe
(tmp_path: Path)
tests/test_harbor_case_id_safety.py:144
Methodtest_consecutive_hyphens
Test consecutive hyphens are rejected by kebab-case pattern.
tests/validators/test_naming_utils.py:30
Methodtest_consecutive_hyphens_rejected
Test validation fails for names with consecutive hyphens.
tests/validators/test_schema.py:199
Functiontest_consecutive_tool_steps_count_each_gap_once
()
tests/tier3/test_trajectory_timing.py:199
Functiontest_consistent_native_neutral_status_is_not_replaced_by_incomplete
(tmp_path)
tests/ci/test_benchmark_reports.py:357
Functiontest_consistent_optional_native_skip_stays_disclosed
(tmp_path)
tests/ci/test_benchmark_reports.py:371
Functiontest_consistent_timing_sources_are_not_flagged
(source)
tests/tier3/test_resource_usage.py:215
Methodtest_constructor_refresh_and_path_lookup_do_not_launch_scanner
(self, scanner_bin)
tests/utils/test_tool_runner.py:535
Methodtest_constructs_openai_client
(self, monkeypatch: pytest.MonkeyPatch)
tests/inference/test_llm_client.py:130
Methodtest_consumer_in_the_same_step_cannot_have_seen_the_output
(self)
tests/tier3/test_plugin_signals.py:928
Functiontest_consumer_python_module_cannot_shadow_installed_evaluator
(consumer, tmp_path)
tests/ci/test_run_tiers.py:414
Methodtest_contained_agent_directory_alias_is_skipped_and_target_is_collected_once
( self, skill_root: Path, )
tests/deduplication/utils/test_skill_collector.py:338
Methodtest_contained_directory_alias_is_skipped_and_target_discovered_once
(self, tmp_path: Path, write_skill)
tests/embedding/test_extractor.py:541
Functiontest_contained_directory_alias_target_rejects_absolute_link_text
(tmp_path: Path)
tests/utils/test_tier2_paths.py:77
Functiontest_contained_directory_alias_target_rejects_hard_linked_alias
(tmp_path: Path)
tests/utils/test_tier2_paths.py:91
Functiontest_contained_directory_alias_target_rejects_root_escape_and_cycle_claims
( tmp_path: Path, relative: str, target: str )
tests/utils/test_tier2_paths.py:69
Functiontest_contained_directory_alias_target_rejects_unreadable_link_text
(tmp_path: Path)
tests/utils/test_tier2_paths.py:84
Functiontest_contained_directory_alias_target_returns_lexical_in_root_claim
( tmp_path: Path, relative: str, target: str, expected: str )
tests/utils/test_tier2_paths.py:49
Functiontest_contained_filtered_transfer_propagates_directory_traversal_failure
( local_output_environment: SkillEvaluatorLocalEnvironment, tmp_path: Path, monkeypatch: pytest.MonkeyPatc
tests/test_harbor_local_environment.py:834
Functiontest_contained_invalid_name_charset_blocked
()
tests/validators/test_mcp_static.py:48
Methodtest_contained_plugin_merges_bundled_skill_schema_findings
(self, tmp_path: Path)
tests/validators/test_plugin_schema.py:141
Methodtest_contained_plugin_requires_non_empty_name
(self, tmp_path: Path)
tests/validators/test_plugin_schema.py:120
Methodtest_contained_plugin_with_name_passes
(self, tmp_path: Path)
tests/validators/test_plugin_schema.py:108
Methodtest_contained_plugin_with_valid_bundled_skill_passes
(self, tmp_path: Path)
tests/validators/test_plugin_schema.py:155
Functiontest_contained_transfer_linked_destination_failure_cleans_staging_and_preserves_output
( local_output_environment: SkillEvaluatorLocalEnvironment, tmp_path: Path, method: str, kind: str )
tests/test_harbor_local_environment.py:781
Functiontest_contained_transfer_refuses_outside_source_before_destination_or_staging_mutation
( local_output_environment: SkillEvaluatorLocalEnvironment, tmp_path: Path, method: str, spell
tests/test_harbor_local_environment.py:628
Functiontest_container_malformed_registry_port_is_never_contacted
(audit, tmp_path)
tests/validators/test_private_dependency_audit.py:396
Methodtest_container_revisions_are_compared_exactly
Only the source revision is case-folded, as the runbook says.
tests/test_benchmark_source_provenance.py:670
Functiontest_containing_set_matches_pairwise_directory_matching
(seed)
tests/ci/test_select_changed_skills.py:252
Methodtest_contains_chunk_details
(self, make_cluster)
tests/deduplication/intra_skill/test_intra_llm_analyzer.py:357
Methodtest_contains_classify_instruction
(self, make_cluster)
tests/deduplication/intra_skill/test_intra_llm_analyzer.py:371
Functiontest_contains_credential_value_detects_embedded_identity_tokens
(value: str)
tests/test_safety_boundaries.py:60
← previousnext →14,401–14,500 of 24,741, ranked by callers