MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

Functiontest_provider_environment_refreshes_adc_token_when_credential_env_is_adc
Verify runner re-acquires a fresh ADC token when provider uses ADC credential_env.
tests/test_harbor_gke_mode.py:906
Functiontest_provider_error_is_saved_as_an_error_attempt
(tmp_path: Path, monkeypatch: pytest.MonkeyPatch)
tests/tier3/test_judge_contracts.py:854
Methodtest_provider_error_on_the_retry_reports_unavailable
(self, tmp_path)
tests/validators/test_rubric_eval.py:1605
Methodtest_provider_errors_are_not_retried
(self, tmp_path, error, diagnostic)
tests/validators/test_rubric_eval.py:1564
Methodtest_provider_failure_keeps_static_findings_and_failed_verdict
( self, mock_tools, sample_skill_dir: Path, )
tests/validators/test_security.py:7055
Functiontest_provider_failure_retains_selection_coverage
(tmp_path: Path, monkeypatch)
tests/embedding/test_collection_coverage.py:168
Functiontest_provider_http_status_survives_sdk_wrapping
( isolated_provider_environment, monkeypatch, provider, status, record_property, )
tests/test_retry_provider_transport.py:177
Functiontest_provider_ids_match_the_skillspector_bridge
()
tests/test_skillspector_provenance.py:262
Functiontest_provider_jobs_require_proven_write_permission
(tmp_path, monkeypatch, stage, permission, event_name)
tests/ci/test_github_admission_workflow.py:97
Methodtest_provider_message_needs_llm_stage_exit_2
( self, mock_tools, use_llm: bool, exit_code: int, expected: str,
tests/validators/test_security.py:7356
Methodtest_provider_message_needs_skillspector_2_12_0
Before 2.12.0 the reason covered only a connection error left after the provider SDK's retries.
tests/validators/test_security.py:7140
Methodtest_provider_only_failure_is_incomplete_with_retry_advice
( self, mock_tools, old_scanner: bool, sample_skill_dir: Path, )
tests/validators/test_security.py:7002
Methodtest_provider_outage_with_a_baseline_gets_only_the_retry_advice
( self, mock_tools, sample_skill_dir: Path, tmp_path: Path, )
tests/validators/test_security.py:7033
Methodtest_provider_prefixed_gpt5_models_use_completion_tokens
(self, model: str)
tests/inference/test_llm_client.py:294
Functiontest_provider_record_without_provenance_is_reported
()
tests/test_skillspector_provenance.py:254
Functiontest_provider_recovers_after_transient_loopback_failures
( isolated_provider_environment, monkeypatch, provider, record_property, )
tests/test_retry_provider_transport.py:252
Functiontest_provider_steps_cannot_run_when_post_approval_head_check_failed
()
tests/ci/test_all_tier_workflows.py:212
Functiontest_provider_stub_fails_before_any_request_when_the_sdk_client_is_not_httpx
( monkeypatch: pytest.MonkeyPatch, )
tests/tier3/test_ragas_judge_evidence.py:807
Functiontest_proxy_settings_leave_the_provider_stub_answering
A proxy setting must not break the stub or route the verifier's request around it.
tests/tier3/test_ragas_judge_evidence.py:787
Functiontest_pruned_non_best_agent_reports_declared_trials_and_reason
()
tests/reporting/test_dashboard_data.py:554
Functiontest_public_benchmark_cards_do_not_name_custom_backends
(acme: types.ModuleType)
tests/test_custom_execution_backends.py:437
Functiontest_public_cli_exposes_both_tier_two_workflows_without_a_service
()
tests/test_oss_packaging.py:788
Functiontest_public_collection_fails_closed_on_linked_step_sidecar_boundaries
( tmp_path: Path, linked_component: str, )
tests/test_issue55_collector_truth.py:2398
Functiontest_public_collection_fails_closed_when_present_sidecar_cannot_be_read
( tmp_path: Path, sidecar_kind: str, )
tests/test_issue55_collector_truth.py:2333
Functiontest_public_collection_fails_closed_when_sidecar_candidate_limit_is_exceeded
( tmp_path: Path, total_candidate_count: int, expected_status: str, )
tests/test_issue55_collector_truth.py:2222
Functiontest_public_collection_fails_closed_when_step_entry_scan_limit_is_exceeded
( tmp_path: Path, entry_count: int, expected_status: str, )
tests/test_issue55_collector_truth.py:2268
Functiontest_public_collection_fails_closed_when_step_sidecars_cannot_be_scanned
( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, )
tests/test_issue55_collector_truth.py:2300
Functiontest_public_collection_rejects_symlinked_trial_root_without_reading_external_artifacts
(tmp_path: Path)
tests/test_issue55_collector_truth.py:2514
Functiontest_public_command_and_pinning_checks_report_into_one_list
()
tests/validators/test_mcp_static.py:204
Functiontest_public_config_accepts_runtime_controls
(tmp_path: Path)
tests/test_tier3_public_runtime.py:73
Functiontest_public_config_still_rejects_sandbox_policy
The public engine has no consumer for a config-level sandbox policy.
tests/test_tier3_public_runtime.py:106
Functiontest_public_config_validates_runtime_control_values
(tmp_path: Path, key: str, value: str)
tests/test_tier3_public_runtime.py:97
Functiontest_public_distributions_include_nvidia_build_runtime_bridges
(tmp_path: Path)
tests/test_oss_packaging.py:604
Functiontest_public_docker_docs_explain_project_dotenv_rejection
()
tests/test_oss_packaging.py:501
Functiontest_public_docker_image_uses_only_public_dependencies
()
tests/test_oss_packaging.py:663
Functiontest_public_docs_declare_support_and_security_sections
()
tests/test_oss_packaging.py:559
Functiontest_public_docs_explain_the_single_nvidia_credential_skillspector_path
()
tests/test_oss_packaging.py:387
Functiontest_public_docs_have_no_personal_staging_ownership
()
tests/test_oss_packaging.py:730
Functiontest_public_docs_match_harbor_environment_and_kwarg_contract
()
tests/test_oss_packaging.py:317
Functiontest_public_docs_show_external_nvidia_build_harness_paths_only
()
tests/test_oss_packaging.py:845
Functiontest_public_docs_show_tier_two_collection_and_catalog_workflows
()
tests/test_oss_packaging.py:804
Functiontest_public_exec_streams_through_harbor_scoped_output_callback
( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, secret: str, )
tests/test_harbor_secure_docker_environment.py:1639
Functiontest_public_extras_exclude_internal_runtime_dependencies
()
tests/test_oss_packaging.py:364
Functiontest_public_extras_use_public_dependency_sources
()
tests/test_oss_packaging.py:129
Functiontest_public_grading_mode_normalizes_for_the_existing_engine
(tmp_path: Path)
tests/test_evals_config_branding.py:22
Functiontest_public_names_resolving_to_private_addresses_are_refused
()
tests/tier3/test_mcp_proof.py:415
Functiontest_public_package_has_no_private_package_indexes
()
tests/test_oss_packaging.py:123
Functiontest_public_package_metadata_has_no_personal_email_addresses
()
tests/test_oss_packaging.py:782
Functiontest_public_policy_serialization_has_stable_digest
()
tests/validators/test_policy.py:212
Functiontest_public_product_branding_uses_the_canonical_name
()
tests/test_oss_packaging.py:1048
Functiontest_public_provider_defaults_are_normalized_for_each_agent
( provider: ProviderConfig, agent: str, expected: str, )
tests/test_provider_agent_routing.py:65
Functiontest_public_provider_matches_shared_contract_fixture
()
tests/test_provider_config.py:332
Functiontest_public_provider_only_entry_passes
()
tests/validators/test_mcp_static.py:36
Functiontest_public_publication_does_not_require_pages_visibility_endpoint
()
tests/ci/test_publish_reports.py:1700
Functiontest_public_readme_is_a_concise_docs_landing_page
()
tests/test_oss_packaging.py:569
Functiontest_public_registration_and_pinned_docker_command
(tmp_path)
tests/test_hermes_agent.py:100
Functiontest_public_release_has_no_internal_repository_metadata
()
tests/test_oss_packaging.py:711
Functiontest_public_release_includes_plc_template_work_products
()
tests/test_oss_packaging.py:657
Functiontest_public_repository_is_recognized_and_default_url_is_preserved
()
tests/test_ci_onboarding_pages.py:39
Functiontest_public_secure_copy_api_documents_same_uid_boundary
()
tests/test_harbor_secure_copy.py:50
Functiontest_public_slim_docker_image_uses_only_distribution_dependencies
()
tests/test_oss_packaging.py:677
Functiontest_public_source_files_fall_back_without_git_metadata
(tmp_path: Path, monkeypatch)
tests/test_oss_packaging.py:695
Functiontest_public_sources_exclude_retired_internal_runtime_paths
()
tests/test_oss_packaging.py:374
Functiontest_public_sources_use_the_public_nvidia_build_contract
()
tests/test_oss_packaging.py:597
Functiontest_public_starter_dataset_passes_strict_validate
(tmp_path: Path)
tests/tier3/test_dataset_baseline_fairness.py:235
Functiontest_public_tier3_applicability_survives_retention
()
tests/reporting/test_retained_tiers.py:154
Functiontest_public_token_accounting_folds_generic_methods_but_keeps_specific_notices_beside_rows
()
tests/reporting/test_report_clarity.py:239
Functiontest_public_tree_has_no_legacy_root_version_module
()
tests/test_oss_packaging.py:726
Functiontest_public_url_rejects_empty_or_repeated_trailing_slashes
(url)
tests/test_ci_onboarding_pages.py:141
Functiontest_publication_and_cleanup_failures_keep_the_run_status
(tmp_path: Path)
tests/tier3/test_plugin_execution.py:98
Functiontest_publication_eligibility_sits_next_to_the_verdict
()
tests/reporting/test_report_redesign.py:328
Functiontest_publication_evidence_parser_fails_closed
(mutation: dict[str, object])
tests/test_publication_evidence.py:59
Functiontest_publication_metadata_rejects_huge_text_before_semantic_normalization
( monkeypatch: pytest.MonkeyPatch, )
tests/reporting/test_advisory_skip_verdict.py:3973
Functiontest_publication_metadata_uses_the_same_strict_json_policy
(tmp_path)
tests/ci/test_github_api.py:154
Functiontest_publication_status_matrix_is_consistent_across_reporters
( verdict: str, score: float, expected_status: str, eligible: bool, )
tests/reporting/test_advisory_skip_verdict.py:1859
Functiontest_publication_target_does_not_trust_scandir_identity_metadata
( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, )
tests/test_publication_identity.py:665
Functiontest_publication_target_normalizes_the_visible_skill_name
(tmp_path: Path)
tests/test_publication_identity.py:655
Functiontest_publication_target_uses_filesystem_entry_spelling_for_case_alias
(tmp_path: Path)
tests/test_publication_identity.py:716
Functiontest_publication_tier1_lineup_matches_the_default_validation_run
()
tests/test_publication_evidence.py:415
Functiontest_publication_uses_triggering_attempt_even_after_a_rerun
(tmp_path, latest_status)
tests/ci/test_publish_reports.py:611
Functiontest_publish_docs_installs_the_fern_cli_from_the_committed_lockfile
The secret-bearing job runs a CLI whose whole tree was reviewed, not resolved. ``npm ci`` reproduces ``fern/package-lock.json`` exactly -- every
tests/test_ci_workflows.py:578
Functiontest_publish_keeps_history_and_updates_only_current_main
(tmp_path)
tests/ci/test_publish_reports.py:887
Functiontest_publish_latest_results_is_atomic_under_concurrency
(tmp_path: Path)
tests/test_results_location.py:386
Functiontest_publish_latest_results_preserves_real_directory_and_cleans_temp_link
(tmp_path: Path)
tests/test_results_location.py:405
Functiontest_publish_step_reports_whether_it_pushed_new_content
(tmp_path, diffs, pushes, committed, commits)
tests/ci/test_publisher_workflow.py:339
Functiontest_published_entry_redacts_token_without_changing_verifier_entry
(canary)
tests/tier3/test_private_plugin_canary.py:61
Functiontest_published_execution_error_without_raw_json_gets_a_diagnostic_page
(tmp_path)
tests/ci/test_native_reports.py:125
Functiontest_published_execution_errors_are_redacted_bounded_and_counted
()
tests/test_harbor_runner_status.py:25
Functiontest_published_execution_errors_bound_serialized_multibyte_sample
()
tests/test_harbor_runner_status.py:39
Functiontest_published_execution_errors_count_distinct_details_beyond_truncation
()
tests/test_harbor_runner_status.py:57
Functiontest_published_json_omits_absolute_host_target_without_changing_evidence
( tmp_path, target, path_override, published )
tests/ci/test_secret_redaction.py:889
Functiontest_published_json_redacts_known_credentials_in_repository_relative_target_override
(tmp_path)
tests/ci/test_secret_redaction.py:911
Functiontest_published_paired_runtime_does_not_require_typescript_source
(tmp_path: Path, with_setup: bool)
tests/test_openclaw_plugin_discovery.py:358
Methodtest_published_pass_needs_the_identity
(self, tmp_path: Path, verdict_line: str)
tests/test_benchmark_source_provenance.py:924
Functiontest_published_report_preserves_recorded_llm_request
(tmp_path, enabled)
tests/ci/test_publish_reports.py:1777
Functiontest_publisher_api_responses_use_the_smaller_shared_cap
()
tests/ci/test_github_api.py:131
Functiontest_publisher_requires_explicit_trust_configuration
(tmp_path, monkeypatch, variable)
tests/ci/test_publish_reports.py:1056
Functiontest_publisher_selects_the_evidence_artifact_not_the_full_reports
()
tests/ci/test_publish_reports.py:714
Functiontest_publisher_uses_verified_node24_action_releases
()
tests/ci/test_publisher_workflow.py:75
Methodtest_pull_request_merge_ref_uses_local_head_without_valid_github_sha
A missing or malformed workflow SHA cannot replace the checked-out revision.
tests/utils/test_helpers.py:316
← previousnext →20,501–20,600 of 24,741, ranked by callers