MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

Functiontest_all_report_formats_keep_version_delta_separate
(evaluation)
tests/test_version_comparison.py:228
Functiontest_all_reporters_fail_closed_across_conflicting_complete_tier3_payloads
( verdict: str, score: float, )
tests/reporting/test_advisory_skip_verdict.py:1242
Functiontest_all_reporters_prefer_publication_complete_payload_over_scored_decoy
()
tests/reporting/test_advisory_skip_verdict.py:1279
Functiontest_all_reporters_select_newest_complete_tier3_payload_before_score
()
tests/reporting/test_advisory_skip_verdict.py:838
Functiontest_all_reporters_select_same_complete_tier3_payload_independent_of_order
()
tests/reporting/test_advisory_skip_verdict.py:821
Functiontest_all_reporters_tolerate_non_mapping_agent_payload
()
tests/reporting/test_advisory_skip_verdict.py:1311
Methodtest_all_required_sections_present_passes
Test validation passes when all body requirements are met.
tests/validators/test_schema.py:765
Functiontest_all_six_formats_compose_static_and_runtime_without_losing_shared_facts
()
tests/reporting/test_private_runtime_render.py:101
Functiontest_all_six_private_surfaces_carry_the_same_static_facts
()
tests/reporting/test_private_static_reports.py:238
Functiontest_all_skills_run_after_failure_and_scanner_stdout_is_not_printed
(tmp_path, capsys)
tests/ci/test_run_tier1.py:372
Functiontest_all_telemetry_text_is_redacted_before_shortening_and_controls_are_removed
()
tests/reporting/test_telemetry_view.py:316
Functiontest_all_tier_ci_tier3_dataset_check_is_strict_about_fairness_warnings
(tmp_path: Path, monkeypatch)
tests/tier3/test_dataset_baseline_fairness.py:348
Functiontest_all_tier_only_init_options_require_tiers
(repository, release_file)
tests/test_ci_onboarding_all_tiers.py:1020
Functiontest_all_tier_preview_lists_proposed_files_without_show_files
(repository, all_release)
tests/test_ci_onboarding_all_tiers.py:1056
Functiontest_all_tier_receipt_caller_path_is_explicit_and_legacy_default_is_unchanged
(monkeypatch)
tests/ci/test_feedback.py:1661
Functiontest_all_tier_receipt_parent_is_created_before_saving
(repository, all_release, tmp_path)
tests/test_ci_onboarding_all_tiers.py:1049
Functiontest_all_tier_runtimes_sync_the_reviewed_lock_with_pinned_uv
()
tests/ci/test_all_tier_workflows.py:336
Functiontest_all_tiers_conflict_does_not_partially_save
(repository, all_release, content)
tests/test_ci_onboarding_all_tiers.py:204
Functiontest_all_tiers_preview_offline_and_local_options
(repository, all_release, monkeypatch)
tests/test_ci_onboarding_all_tiers.py:158
Functiontest_all_tiers_requires_capability_instead_of_using_legacy_refs
(repository, release_file)
tests/test_ci_onboarding_all_tiers.py:188
Functiontest_all_tiers_save_github_is_minimal_pinned_and_idempotent
(repository, all_release)
tests/test_ci_onboarding_all_tiers.py:80
Functiontest_all_tiers_save_preserves_reusable_feedback_workflow
(repository, all_release, reusable_feedback)
tests/test_ci_onboarding_all_tiers.py:106
Functiontest_all_tiers_without_release_uses_packaged_candidate_descriptor
(repository, tmp_path, monkeypatch)
tests/test_ci_onboarding_all_tiers.py:663
Methodtest_all_validators_handle_missing_skill_md
Test all validators handle directories without SKILL.md.
tests/validators/test_security_validators.py:935
Functiontest_all_values_use_subprocess_env_including_empty_and_special_values
()
tests/test_harbor_secure_docker_environment.py:7117
Functiontest_allow_observed_success
(workspace)
tests/test_trace_import.py:248
Functiontest_allowed_alternate_activation_is_optional
(tmp_path: Path, alias: str, skill: str | None)
tests/reporting/test_activation_validation.py:151
Methodtest_allowed_dirs_extended_via_env
``SKILLEVALUATOR_SCHEMA_ALLOWED_DIRS`` extends the allowed skill-root dirs. A consumer with a non-canonical directory (e.g. ``data/``) should
tests/validators/test_schema.py:865
Functiontest_allowed_root_escape_and_symlink_are_rejected
(tmp_path: Path)
tests/test_harbor_secure_copy.py:264
Methodtest_allowed_tools_field_accepted
Test that the optional 'allowed-tools' field is accepted per agentskills.io spec.
tests/validators/test_schema.py:731
Functiontest_allowed_tools_flag_flags_powershell_grant
()
tests/validators/test_plugin_permissions.py:64
Functiontest_allowlist_rejects_anchor_ids_that_match_a_denied_rule
(tmp_path: Path)
tests/test_oss_boundary.py:406
Functiontest_allowlist_rejects_duplicate_anchor_ids_across_paths
(tmp_path: Path)
tests/test_oss_boundary.py:441
Functiontest_allowlist_rejects_exceptions_outside_negative_tests
(tmp_path: Path)
tests/test_oss_boundary.py:493
Functiontest_allowlist_rejects_expired_negative_test_exception
(tmp_path: Path)
tests/test_oss_boundary.py:519
Functiontest_allowlist_rejects_legacy_line_number_schema
(tmp_path: Path)
tests/test_oss_boundary.py:380
Functiontest_allowlist_rejects_missing_reason_and_unknown_metadata
(tmp_path: Path)
tests/test_oss_boundary.py:415
Functiontest_allowlist_rejects_noncanonical_test_paths_and_anchor_ids
( tmp_path: Path, allow_path: str, anchor: str, )
tests/test_oss_boundary.py:477
Functiontest_allowlist_revocation_never_sends_credentials_to_proxy
(monkeypatch, revoke_before_policy_check: bool)
tests/test_model_catalog.py:813
Functiontest_allowlisted_catalog_transport_bypasses_environment_proxy
The bearer must not be handed to an inherited proxy for a host we accept.
tests/test_model_catalog.py:778
Functiontest_allowlisting_a_host_does_not_allow_a_different_one
(monkeypatch)
tests/test_model_catalog.py:722
Methodtest_allows_placeholder_secrets
Test that placeholder secrets are allowed.
tests/validators/test_security.py:6786
Methodtest_allows_rfc1918_private_ips
Test that RFC1918 private IPs are allowed.
tests/validators/test_security.py:6801
Methodtest_already_model_returns_as_is
Test that model instance is returned unchanged.
tests/models/test_field_validators.py:58
Methodtest_alternatives_and_violations
(self)
tests/tier3/test_plugin_signals.py:885
Methodtest_always_apply_is_required
Test that alwaysApply field is required for Rules.
tests/validators/test_rules_schema.py:239
Methodtest_always_apply_is_required
Test that alwaysApply field is required for Workflows.
tests/validators/test_workflows_schema.py:341
Functiontest_ambiguous_codex_items_do_not_borrow_identity_or_success
(server, output)
tests/tier3/test_plugin_evidence_review.py:55
Functiontest_ambiguous_init_does_not_promote_staging_without_a_file_census
()
tests/tier3/test_native_review_fixes.py:133
Methodtest_ambiguous_object_does_not_hide_another_object
(self, monkeypatch: pytest.MonkeyPatch)
tests/inference/test_llm_client.py:954
Methodtest_ambiguous_object_is_rejected
( self, monkeypatch: pytest.MonkeyPatch, prefix: str, payload: str, reason: str )
tests/inference/test_llm_client.py:944
Methodtest_ambiguous_or_mismatched_revisions_are_dropped
A short prefix can grow a second match, and a digest must match its own algorithm.
tests/test_benchmark_source_provenance.py:132
Functiontest_ambiguous_or_other_destructive_commands_remain_findings
(module, command)
tests/tier3/test_cache_cleanup_scoring.py:72
Functiontest_ambiguous_or_unrecognized_metadata_retains_ordinary_email_policy
(tmp_path, frontmatter)
tests/validators/test_pii_ownership.py:91
Methodtest_ambiguous_reference_stays_incomplete
(self, mock_tools, tmp_path: Path, version: str)
tests/validators/test_security.py:9758
Methodtest_ambiguous_revision_does_not_satisfy_a_pass
(self, tmp_path: Path, revision: str)
tests/test_benchmark_source_provenance.py:1052
Functiontest_ambiguous_same_server_codex_outcomes_remain_unknown_after_overlay
(tmp_path)
tests/tier3/test_plugin_evidence_review.py:333
Functiontest_ambiguous_write_observation_remains_wrapper_level_evidence
(behavior_builder, refs_builder)
tests/tier3/test_codex_tool_call_normalization.py:947
Functiontest_an_agent_without_scores_still_gets_canonical_case_rows
()
tests/tier3/test_token_usage.py:723
Functiontest_an_ansi_c_quote_honours_its_escapes_when_finding_where_commands_end
``$'...'`` closes on ``'`` as ``'...'`` does, but a backslash escapes the character after it: in ``$'it\\'s # kept'`` the ``#`` is quoted and star
tests/tier3/test_script_invocation_evidence.py:1999
Functiontest_an_argument_that_may_reach_httpie_as_a_header_file_item
(word, expected)
tests/tier3/test_network_client_mentions.py:535
Functiontest_an_argv_element_that_a_masked_flag_value_cuts_keeps_its_password_masked
( text: str, expected: str, judge: bool )
tests/tier3/test_security_finding_evidence.py:3128
Functiontest_an_argv_flag_value_that_names_a_command_keeps_its_attached_password_masked
(flag: str, argv: str)
tests/tier3/test_security_finding_evidence.py:2470
Functiontest_an_arithmetic_command_is_not_a_pair_of_subshells
``((f=x))`` at command position is arithmetic and runs in the current shell: the value it gives a variable is a number, never a script path; on
tests/tier3/test_script_invocation_evidence.py:1495
Functiontest_an_arithmetic_shift_is_not_a_heredoc
``<<`` inside an arithmetic expansion shifts, so no body follows it.
tests/tier3/test_script_invocation_evidence.py:521
Functiontest_an_arm_that_died_keeps_its_tokens_on_the_canonical_case
()
tests/tier3/test_token_usage.py:652
Functiontest_an_arm_the_run_mode_leaves_out_did_not_run
()
tests/tier3/test_token_usage.py:468
Functiontest_an_arm_without_a_known_run_status_gets_no_change
(status: str | None)
tests/tier3/test_token_usage.py:562
Functiontest_an_assignment_before_a_command_is_that_commands_environment
``f=run.py python3 "$f"`` expands ``$f`` before the assignment applies, and the assignment is the command's environment only, so nothing after it
tests/tier3/test_script_invocation_evidence.py:1601
Functiontest_an_assignment_ending_in_an_operator_is_not_a_redirection
An unquoted redirection operator is a word of its own, so only such a word, with a descriptor written flush against it (``2>``), takes the next
tests/tier3/test_script_invocation_evidence.py:1955
Functiontest_an_assignment_or_builtin_that_fails_where_the_shell_stops_ends_the_credit
A value that is not a number, given to a ``-i`` name, is an error, and bash exits there, as every shell with ``typeset`` does for most forms (
tests/tier3/test_script_invocation_evidence.py:2233
Functiontest_an_attached_descriptor_before_the_script_is_a_redirection
The descriptor and its operator stay together through tokenizing, so a redirection written before the script never stands where the script is
tests/tier3/test_script_invocation_evidence.py:966
Functiontest_an_attempt_that_never_ran_leaves_the_arm_partial
(pass_at_k: dict[str, Any])
tests/tier3/test_token_usage.py:286
Functiontest_an_authorization_env_name_needs_a_value_shaped_like_key_material
()
tests/validators/test_url_policy.py:535
Functiontest_an_authorization_header_names_a_credential_whatever_its_scheme
Regression: only 'Bearer' and 'Basic' payloads counted, so 'Authorization: Token <hex>' was no credential.
tests/validators/test_url_policy.py:529
Functiontest_an_empty_load_failure_map_changes_nothing
()
tests/tier3/test_private_ladder_stats.py:193
Functiontest_an_empty_loop_skipped_inside_a_subshell_still_closes_the_subshell
Skipping the body of an empty loop must not skip the ``)`` that ends the group it sits in: the group's bindings stay inside the group, and the
tests/tier3/test_script_invocation_evidence.py:1404
Functiontest_an_error_line_appended_after_a_long_answer_still_fails
()
tests/tier3/test_f90_mcp_failure_text.py:213
Functiontest_an_excluded_case_is_out_of_every_step_and_the_glue
()
tests/tier3/test_private_ladder_stats.py:312
Methodtest_an_expectation_requires_a_card
(self, tmp_path: Path, capsys: pytest.CaptureFixture[str])
tests/test_benchmark_source_provenance.py:778
Functiontest_an_expected_case_no_arm_scored_is_listed_with_every_arm
()
tests/tier3/test_private_ladder_stats.py:152
Functiontest_an_explicit_empty_environment_is_not_the_host_environment
( fake_mcp: str, monkeypatch: pytest.MonkeyPatch )
tests/tier3/test_mcp_proof.py:297
Methodtest_an_implementation_revision_satisfies_a_pass
An evaluator built from a checkout has no image digest to record.
tests/test_benchmark_source_provenance.py:1027
Functiontest_an_interpreter_reading_its_program_from_a_pipe_is_unresolved
``cat run.py | python3`` runs the script and ``cat run.py | wc -l`` does not; an interpreter with no script, no inline code and no terminal option
tests/tier3/test_script_invocation_evidence.py:1318
Methodtest_an_invalid_field_does_not_discard_the_other_channel
(self)
tests/test_benchmark_source_provenance.py:806
Functiontest_an_invocation_inside_a_heredoc_body_is_still_data
(check, command)
tests/tier3/test_script_invocation_evidence.py:506
Functiontest_an_invocation_inside_an_unmodelled_command_is_unresolved
The set of commands that run another command is open, so it is not listed. A command with no grammar of its own that carries an interpreter runni
tests/tier3/test_script_invocation_evidence.py:675
Functiontest_an_invocation_quoted_as_data_is_not_an_invocation
Command text that only describes running the script is not evidence. A heredoc body in particular is the operand's data, not further commands,
tests/tier3/test_script_invocation_evidence.py:303
Methodtest_an_mcp_editor_view_does_not_write_the_artifact
(self)
tests/tier3/test_plugin_signals.py:298
Functiontest_an_mcp_tool_called_through_the_shell_counts
check-19 e06: publish_release sent as JSON-RPC to the server program through Bash/exec_command.
tests/tier3/test_f90_signals_conflict.py:180
Functiontest_an_off_origin_sse_endpoint_fails_fast
(fake_mcp: str)
tests/tier3/test_mcp_proof.py:594
Functiontest_an_omitted_cache_counter_is_zero_and_atif_names_are_read
()
tests/tier3/test_token_usage.py:94
Functiontest_an_operand_written_flush_against_its_descriptor_tokenizes_as_the_spaced_form
``0<run.py`` and ``0< run.py`` are the same redirection, so they must score the same. The descriptor stays with its operator, and the operand
tests/tier3/test_script_invocation_evidence.py:1068
Methodtest_an_oversized_limit_does_not_stop_the_scan
(self, mock_tools, tmp_path: Path)
tests/validators/test_skillspector_reference_diagnostics.py:441
Functiontest_an_oversized_response_ends_the_probe_at_once_without_mcp_tracebacks
( fake_mcp: str, caplog: pytest.LogCaptureFixture )
tests/tier3/test_mcp_proof.py:580
Functiontest_an_underscored_flag_or_config_value_is_masked_whole_unless_a_name_in_it_runs_past_it
( text: str, expected: str )
tests/tier3/test_security_finding_evidence.py:2905
Functiontest_an_underscored_flag_takes_the_next_word_as_its_dashed_spelling_does
( copy: str, command: str, underscored: str, dashed: str )
tests/tier3/test_security_finding_evidence.py:2239
Functiontest_an_underscored_flag_value_the_assignment_rule_would_not_mask_whole_is_masked_as_its_dashed_spelling
( flag: str, value: str )
tests/tier3/test_security_finding_evidence.py:2925
Functiontest_an_underscored_flag_value_with_an_escaped_quote_assignment_is_masked_whole
( copy: str, text: str, expected: str )
tests/tier3/test_security_finding_evidence.py:4199
Functiontest_an_unmodelled_command_that_only_reads_the_script_is_still_zero
Carrying the script as an argument is not carrying an invocation of it.
tests/tier3/test_script_invocation_evidence.py:692
← previousnext →12,601–12,700 of 24,741, ranked by callers