MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

Functiontest_a_case_that_a_run_budget_cut_short_expects_only_the_attempts_it_admitted
()
tests/tier3/test_token_usage.py:342
Functiontest_a_changed_byte_makes_that_piece_a_mismatch
( tmp_path: Path, loose, staged_rel: str, expected_mismatches )
tests/tier3/test_private_loose_source.py:347
Functiontest_a_clean_four_arm_run_gives_every_step_and_the_bundle
()
tests/tier3/test_private_ladder_stats.py:214
Functiontest_a_command_completed_before_a_refused_line_keeps_its_credit
bash, bash --posix, dash, ksh, mksh and busybox ash parse and run a text one complete command at a time, so the first command, completed on a
tests/tier3/test_script_invocation_evidence.py:2405
Functiontest_a_command_line_never_borrows_a_notice_written_above_it
(text: str)
tests/tier3/test_issue192_security_false_positives.py:326
Functiontest_a_command_password_after_a_value_less_flag_does_not_hide_a_later_credential
(flag: str, text: str)
tests/tier3/test_security_finding_evidence.py:2449
Functiontest_a_command_recorded_under_raw_is_still_checked
(module)
tests/tier3/test_tool_productivity.py:452
Functiontest_a_command_that_only_assigns_or_reads_runs_nothing
(check, command)
tests/tier3/test_script_invocation_evidence.py:546
Functiontest_a_command_that_only_names_a_network_client_is_not_exfiltration
(run, cmd)
tests/tier3/test_network_client_mentions.py:150
Functiontest_a_comment_apostrophe_does_not_run_a_quoted_target_on
()
tests/tier3/test_security_home_paths.py:821
Functiontest_a_comment_line_ending_in_a_backslash_ends_the_declaration
As in pip's ``join_lines``, a backslash at the end of a comment line does not continue it.
tests/validators/test_dependency_advisory_findings.py:743
Functiontest_a_comment_or_redirection_after_other_interpreters_is_still_remote_code
(interpreter: str, tail: str)
tests/validators/test_plugin_static_risk_review_fixes.py:271
Functiontest_a_comment_or_redirection_after_the_shell_is_still_remote_code
(flags: str, tail: str)
tests/validators/test_plugin_static_risk_review_fixes.py:263
Functiontest_a_complete_loose_arm_is_complete
()
tests/tier3/test_private_ladder_stats.py:540
Functiontest_a_component_reports_its_permission_flags_in_config_order
One walk per config: the findings follow the config, not all bypass flags before all mode flags.
tests/validators/test_plugin_static_risk_review_fixes.py:877
Functiontest_a_compound_word_after_an_assignment_is_refused_text
A compound's opening word is reserved only where a command starts. After an assignment, bash, bash --posix, dash, zsh, ksh, mksh and busybox a
tests/tier3/test_script_invocation_evidence.py:2366
Functiontest_a_config_key_and_separator_keep_their_value_masked_after_a_masked_command_word
( flag: str, command: str, expected: str )
tests/tier3/test_security_finding_evidence.py:2386
Functiontest_a_config_key_with_json_escaped_quotes_masks_its_value
( copy: str, depth: int, command: str, expected: str )
tests/tier3/test_security_finding_evidence.py:3636
Functiontest_a_config_set_command_after_an_escaped_line_break_keeps_its_value_masked
( copy: str, text: str, kept: str )
tests/tier3/test_security_finding_evidence.py:4048
Functiontest_a_config_value_in_a_quoted_string_ends_at_a_shell_operator
(copy: str)
tests/tier3/test_security_finding_evidence.py:4068
Functiontest_a_config_value_with_spaces_and_json_escaped_quotes_is_masked_whole
(copy: str)
tests/tier3/test_security_finding_evidence.py:3646
Methodtest_a_content_digest_is_accepted_as_the_revision
A run with no upstream commit still records an immutable revision.
tests/test_benchmark_source_provenance.py:1196
Methodtest_a_content_digest_revision_is_matched
(self, tmp_path: Path)
tests/test_benchmark_source_provenance.py:685
Functiontest_a_copied_run_after_a_line_escape_and_an_escaped_character_is_masked
(text: str)
tests/evaluation/test_tier3_feedback_chat.py:714
Functiontest_a_counter_the_trajectory_recorded_invalid_is_rejected_although_tokens_leave_it_out
()
tests/tier3/test_token_usage.py:115
Functiontest_a_credential_on_the_line_after_an_unquoted_value_is_masked
(copy: str, next_line: str)
tests/tier3/test_security_finding_evidence.py:3692
Functiontest_a_credential_that_a_masked_flag_value_runs_into_stays_masked
(flag: str, text: str, expected: str)
tests/tier3/test_security_finding_evidence.py:2534
Functiontest_a_custom_basis_keeps_discoverability_and_efficiency
()
tests/tier3/test_private_ladder_stats.py:199
Functiontest_a_cut_case_name_never_becomes_a_second_row_beside_its_dataset_case
()
tests/tier3/test_token_usage.py:634
Functiontest_a_cut_occurrence_list_is_a_lower_bound
(tmp_path: Path, monkeypatch: pytest.MonkeyPatch)
tests/reporting/test_skillspector_reference_summary.py:217
Functiontest_a_damaged_command_is_scored_never_raised
A malformed command gets a score like any other. Deleting one ``;`` from the second base gives ``f=run.py for f in; do :; ...``, which raised
tests/tier3/test_script_invocation_evidence.py:2666
Functiontest_a_declaration_copies_the_value_when_it_runs
``export g=$f`` copies what ``f`` holds when it runs, as ``g=$f`` does; a later ``f=...`` does not reach ``g``. In one declaration of several name
tests/tier3/test_script_invocation_evidence.py:1714
Functiontest_a_deletion_outside_the_trials_own_output_directory_still_scores
(module, calls: list[dict])
tests/tier3/test_issue192_security_false_positives.py:194
Functiontest_a_digest_that_does_not_match_its_reported_value_is_dropped
()
tests/ci/test_secret_redaction.py:400
Functiontest_a_digit_standing_apart_from_its_operator_is_an_operand
``2 > numeric.out`` is an argument ``2`` and a redirection of standard output, not a redirection of standard error: the shell reads a descriptor
tests/tier3/test_script_invocation_evidence.py:940
Functiontest_a_dot_slash_path_whose_dots_stay_inside_does_not_climb_out
()
tests/validators/test_plugin_component_risk.py:517
Functiontest_a_downgraded_passing_scan_does_not_show_the_counts
A policy that lowers the AE1 findings of a complete scan passes it; no report lists the count line.
tests/reporting/test_skillspector_reference_summary.py:248
Methodtest_a_duplicated_line_cannot_match
(self, tmp_path: Path)
tests/test_benchmark_source_provenance.py:647
Functiontest_a_failed_arm_marks_the_ladder_partial_and_names_the_arm
()
tests/tier3/test_private_ladder_stats.py:342
Functiontest_a_failed_attempt_is_counted_for_usage_but_not_as_a_scored_attempt
()
tests/tier3/test_token_usage.py:186
Functiontest_a_failed_attempt_with_a_cut_harbor_trial_name_is_matched_to_its_case
()
tests/tier3/test_token_usage.py:211
Functiontest_a_failed_baseline_arm_keeps_its_usage_but_gets_no_change
()
tests/tier3/test_token_usage.py:539
Functiontest_a_failed_call_does_not_satisfy_must_use
(agent: str, fn: str, result: dict[str, Any])
tests/tier3/test_f90_signals_conflict.py:97
Functiontest_a_failed_call_never_turns_an_unreachable_server_reachable
()
tests/tier3/test_f90_mcp_proof_status.py:92
Functiontest_a_failed_forbidden_call_is_still_an_attempt
()
tests/tier3/test_f90_signals_conflict.py:106
Methodtest_a_failed_read_stays_failed_without_an_explicit_success
(self, observation: str)
tests/tier3/test_plugin_signals.py:506
Methodtest_a_failure_flag_still_beats_a_zero_exit_status
(self)
tests/tier3/test_plugin_signals.py:510
Functiontest_a_field_with_too_many_values_is_reported_instead_of_skipped
()
tests/tier3/test_dataset_baseline_fairness.py:1925
Functiontest_a_file_a_command_body_runs_is_copied
(tmp_path: Path)
tests/tier3/test_private_loose_source.py:878
Functiontest_a_file_a_skills_frontmatter_hook_names_is_copied_to_the_loose_root
(tmp_path: Path)
tests/tier3/test_private_loose_source.py:805
Functiontest_a_file_named_after_an_unexpanded_variable_does_not_clear_unanalyzed
( tmp_path: Path, command: str, decoy: str )
tests/validators/test_plugin_static_risk_review_fixes.py:556
Functiontest_a_final_frame_refused_when_the_run_closes_is_not_sent_with_the_next_run
( monkeypatch: pytest.MonkeyPatch, stage_state: str, )
tests/test_tier3_progress.py:1665
Functiontest_a_final_output_symlink_can_be_unlinked
(security_check)
tests/tier3/test_agent_output_cleanup_scope.py:85
Methodtest_a_finding_without_reasons_counts_without_claiming_one
A named file with no reasons and no ledger rows is counted; the line claims no reason for it.
tests/validators/test_skillspector_reference_diagnostics.py:774
Functiontest_a_flag_inside_a_dashed_flag_value_is_not_read
(text: str, expected: str, judge: bool)
tests/tier3/test_security_finding_evidence.py:3098
Functiontest_a_flag_inside_a_long_underscored_flag_word_is_not_read
( make_text: Any, monkeypatch: pytest.MonkeyPatch )
tests/tier3/test_security_finding_evidence.py:2846
Functiontest_a_flag_inside_an_earlier_flag_value_keeps_its_own_value_masked
()
tests/tier3/test_security_finding_evidence.py:3032
Functiontest_a_flag_value_that_holds_the_colon_of_a_user_value_keeps_that_value_masked
(flag: str)
tests/tier3/test_security_finding_evidence.py:2985
Functiontest_a_fork_head_cannot_use_writer_permissions
(tmp_path, monkeypatch, stage)
tests/ci/test_github_admission_workflow.py:128
Functiontest_a_hash_inside_a_shell_argument_is_not_a_notice_comment
()
tests/tier3/test_license_notice_network_bypasses.py:107
Functiontest_a_header_dump_or_directory_listing_beside_a_download_is_not_an_upload
(run, cmd)
tests/tier3/test_network_client_mentions.py:389
Functiontest_a_here_string_consumes_only_its_operand
Commands after a here-string are still commands.
tests/tier3/test_script_invocation_evidence.py:403
Functiontest_a_here_string_the_shell_cannot_parse_is_never_full_credit
dash and busybox ash have no here-string: ``<<<`` is a syntax error that stops the line holding it, and a compound command spanning lines around
tests/tier3/test_script_invocation_evidence.py:2263
Functiontest_a_heredoc_inside_a_c_payload_keeps_its_body_as_data
A newline inside the quotes of a ``-c`` payload is kept for the payload's shell, so a heredoc there takes its body after its line, as at the top
tests/tier3/test_script_invocation_evidence.py:2292
Functiontest_a_hook_flag_is_counted_once_however_many_findings_raise_it
Regression: URL credentials and a secret header listed 'inline_secret' twice, so by_flag counted 2.
tests/validators/test_plugin_component_risk.py:723
Functiontest_a_hook_script_of_backslashes_at_the_scan_limit_is_scanned_quickly
(tmp_path: Path)
tests/validators/test_plugin_hook_risk_bounds.py:104
Functiontest_a_large_dataset_entry_is_validated_in_bounded_time
(tmp_path: Path)
tests/tier3/test_dataset_baseline_fairness.py:2090
Methodtest_a_larger_budget_never_widens_the_declared_root
(self, tmp_path, monkeypatch, case)
tests/validators/test_rubric_eval.py:627
Methodtest_a_later_record_replaces_the_count
(self)
tests/validators/test_skillspector_reference_diagnostics.py:752
Functiontest_a_legacy_members_baseline_gets_no_ladder
Legacy ``--lift-mode integration``: the only baseline holds the member skills, so there is no no-plugin rung.
tests/tier3/test_private_ladder_stats.py:611
Functiontest_a_license_url_is_not_a_network_client
(module, text: str)
tests/tier3/test_issue192_security_false_positives.py:296
Functiontest_a_line_refused_on_its_own_is_not_credited_before_a_refused_line
A redirection operator without its operand (at the line's end, before ``;``, or before a comment, which a ``#`` starts after an operator too),
tests/tier3/test_script_invocation_evidence.py:2510
Functiontest_a_line_start_credential_in_json_stays_masked_before_a_flag_value_left_to_the_assignment_rule
( command: str, )
tests/tier3/test_security_finding_evidence.py:4302
Functiontest_a_line_start_credential_stays_masked_before_a_flag_value_read_to_the_end_of_a_quoted_line
( head: str, tail: str, nul: bool )
tests/tier3/test_security_finding_evidence.py:4323
Functiontest_a_line_start_credential_stays_masked_before_a_value_left_to_the_assignment_rule
(head: str, tail: str)
tests/tier3/test_security_finding_evidence.py:4271
Functiontest_a_line_with_more_heredocs_than_the_shell_accepts_is_data
Past the heredocs a shell accepts on one line, the line and everything after it are data: bash exits with "maximum here-document count exceeded"
tests/tier3/test_script_invocation_evidence.py:2045
Methodtest_a_list_of_case_ids_without_a_case_noun_is_checked_in_bounded_time
(self)
tests/evaluation/test_judges.py:1334
Functiontest_a_listing_option_given_names_acts_on_them_where_the_shell_does
``export -p`` and ``readonly -p`` given names act on them as they do without ``-p`` in bash, bash --posix, ksh, mksh and busybox ash, which list
tests/tier3/test_script_invocation_evidence.py:1813
Functiontest_a_load_failure_in_any_arm_drops_the_case_from_every_arm
A part Claude Code did not report loaded (and never called) voids the case in all four arms.
tests/tier3/test_private_ladder_stats.py:162
Functiontest_a_local_run_outside_workspace_still_has_its_own_output_directory
()
tests/tier3/test_issue192_security_false_positives.py:221
Functiontest_a_long_digit_run_is_not_a_write
()
tests/tier3/test_security_home_paths.py:827
Functiontest_a_long_file_write_with_a_license_header_is_not_exfiltration
(module, header: str)
tests/tier3/test_issue192_security_false_positives.py:270
Methodtest_a_long_repository_pinned_by_digest_is_accepted
A cap over the whole reference charged the digest to the name and dropped ordinary references.
tests/test_benchmark_source_provenance.py:160
Methodtest_a_long_repository_pinned_by_digest_satisfies_a_pass
The reference is 140 characters, which the old whole-reference cap discarded.
tests/test_benchmark_source_provenance.py:981
Functiontest_a_long_run_of_quoted_tees_is_read_once
()
tests/tier3/test_security_home_paths.py:838
Functiontest_a_long_sed_value_is_not_read_again_for_each_name_in_it
( copy: str, make_text: Any, monkeypatch: pytest.MonkeyPatch )
tests/tier3/test_security_finding_evidence.py:4227
Functiontest_a_long_underscored_flag_word_does_not_read_each_name_in_it
( make_text: Any, monkeypatch: pytest.MonkeyPatch )
tests/tier3/test_security_finding_evidence.py:2814
Functiontest_a_loop_over_an_empty_list_keeps_the_binding_and_runs_no_body
``for f in; do ...; done`` runs zero iterations and never assigns ``f``, so a value it held before still stands for the command after the loop.
tests/tier3/test_script_invocation_evidence.py:1117
Functiontest_a_loop_variable_rebound_without_a_unique_value_is_forgotten
A single-value header binds its variable for its own body. A later header that gives the same variable several values, or none, settles nothin
tests/tier3/test_script_invocation_evidence.py:1010
Methodtest_a_lowered_finding_is_counted_by_its_new_severity
(self)
tests/models/test_result.py:257
Methodtest_a_malformed_recorded_value_never_matches
A present but malformed line is named as such, not reported as a missing line.
tests/test_benchmark_source_provenance.py:612
Functiontest_a_masked_argv_element_does_not_hide_a_login_password_right_after_the_list
(flag: str)
tests/tier3/test_security_finding_evidence.py:2486
Functiontest_a_masked_key_stays_one_masked_value
(text: str, expected: str)
tests/tier3/test_security_finding_evidence.py:1759
Functiontest_a_matcher_regex_is_parsed_once_for_its_scope_and_its_sensitive_tools
()
tests/validators/test_plugin_hook_risk_bounds.py:318
Functiontest_a_missing_arm_leaves_no_case_and_is_named
()
tests/tier3/test_private_ladder_stats.py:355
Functiontest_a_mode_name_claimed_by_two_packages_is_ambiguous
( backend_module: types.ModuleType, monkeypatch: pytest.MonkeyPatch, tmp_path: Path )
tests/test_custom_execution_backends.py:420
Functiontest_a_name_in_an_underscored_flag_value_is_read_on_its_own
(text: str, expected: str)
tests/tier3/test_security_finding_evidence.py:2974
Functiontest_a_name_of_only_slashes_is_left_as_written
()
tests/test_plugin_refs.py:49
Functiontest_a_name_that_a_value_runs_over_past_a_masked_flag_value_keeps_its_own_value_masked
( flag: str, command: str, expected: str )
tests/tier3/test_security_finding_evidence.py:2605
Functiontest_a_name_that_is_no_credential_on_the_line_after_an_unquoted_value_is_not_read
(copy: str)
tests/tier3/test_security_finding_evidence.py:3721
← previousnext →12,101–12,200 of 24,741, ranked by callers