MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 1 callersFunction_state_path
Resolve the task list state file path. Priority: 1. TASK_LIST_STATE_PATH env var (explicit override) 2. WORKSPACE_MOUNT_PATH env var + /t
src/skillevaluator/tier3/reference_skills/task-list/scripts/tasks.py:23
↓ 1 callersFunction_static_analysis
( inventory: PluginInventory, *, allowed_private_hosts: tuple[str, ...], hook_allowed_urls: tuple[str, ...
src/skillevaluator/plugin_inventory.py:307
↓ 1 callersFunction_static_codex_tool_calls
Decode the complete, bounded statement grammar emitted by native Codex.
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:753
↓ 1 callersMethod_static_test_evidence_message
Return the static-test limitation from direct or aggregated results.
src/skillevaluator/reporting/benchmark.py:945
↓ 1 callersFunction_status
``None`` when some trial scored *metric*, else why none did.
src/skillevaluator/tier3/harbor/lift_basis.py:68
↓ 1 callersFunction_status_groups
(judgments: list[dict[str, Any]])
src/skillevaluator/reporting/diagnostic_view.py:20
↓ 1 callersFunction_step
( kind: str, target: str, problem: str, evidence: str, action: str, check: str, strength: str )
src/skillevaluator/tier3/next_steps.py:71
↓ 1 callersFunction_step_fields
A step's reported fields. A case-type group below the pair minimum keeps its estimate but no interval.
src/skillevaluator/tier3/harbor/stats.py:986
↓ 1 callersFunction_step_results
(step: Any)
src/skillevaluator/tier3/plugin_trace_artifacts.py:375
↓ 1 callersFunction_step_tool_calls
(step: Any)
src/skillevaluator/tier3/eval_core/plugin_signals.py:2925
↓ 1 callersFunction_step_value
(trial: Mapping[str, Any])
src/skillevaluator/reporting/trial_charts.py:20
↓ 1 callersFunction_steps_by_case
Mean steps per case; a case with any unmeasured retry stays unavailable.
src/skillevaluator/reporting/trial_charts.py:52
↓ 1 callersFunction_stop
()
src/skillevaluator/tier3/nemo_gym/runner.py:671
↓ 1 callersFunction_stop_detached_gym_processes
(token: str, leader_group: int, *, force: bool = False)
src/skillevaluator/tier3/nemo_gym/runner.py:530
↓ 1 callersMethod_stop_live_after_failure
Stop the live view after a drawing failure and say that the run goes on. The full final frame could fail the same way, so Live keeps the last
src/skillevaluator/tier3/harbor/progress.py:569
↓ 1 callersFunction_stop_on_pass_case
Whether ``trial_id`` is a stop-on-pass attempt of ``case``: ``...-<with|without>-<case>-attemptNNN__...``.
src/skillevaluator/tier3/token_usage.py:162
↓ 1 callersFunction_stopping_frame_reporter
Return a Rich reporter whose stage sequence matches a run stopped by Ctrl+C.
tests/test_tier3_progress.py:707
↓ 1 callersMethod_store_skillspector_metadata
Store the validated SkillSpector report metadata on the result. With *score_message* false, the caller reports the risk score once it
src/skillevaluator/validators/security.py:5506
↓ 1 callersMethod_stream_keeps_refused_writes
Whether the console's stream keeps a write the terminal refused, as standard error's buffer does.
src/skillevaluator/tier3/harbor/progress.py:623
↓ 1 callersFunction_strict_bool
(value: Any)
src/skillevaluator/evaluation/insights_judge.py:382
↓ 1 callersFunction_strict_json_numbers
Replace non-finite floats before strict generated-artifact serialization.
src/skillevaluator/tier3/harbor/collector.py:6510
↓ 1 callersFunction_strict_model_unavailable
Return whether a failed strict judge request says the model is missing or not open to the key.
src/skillevaluator/tier3/harbor/templates/eval.py:1104
↓ 1 callersFunction_strict_read_root_variants
Return canonical and visible aliases for strict Seatbelt exceptions. macOS exposes many Homebrew/system paths through firmlinks and symlinks
src/skillevaluator/tier3/harbor/local_sandbox.py:394
↓ 1 callersFunction_string_list
(value: Any, config_path: Path, field: str)
src/skillevaluator/tier3/evals_config.py:819
↓ 1 callersMethod_string_open
Return where the quoted string open at *position* starts (None: none is), as the quote scan reads it. The quotes are read forward once, only
src/skillevaluator/tier3/eval_core/checks.py:4923
↓ 1 callersFunction_string_set
(value: Any)
src/skillevaluator/tier3/plugin_native.py:2586
↓ 1 callersFunction_stringify_tool_result_content
(content: Any)
src/skillevaluator/tier3/eval_core/log_converters.py:39
↓ 1 callersFunction_strings
(value: Any, field: str)
src/skillevaluator/reporting/retained.py:68
↓ 1 callersFunction_strings
(value: Any)
src/skillevaluator/tier3/loose_parts.py:716
↓ 1 callersFunction_strings
(value)
tests/reporting/test_private_runtime_replay.py:77
↓ 1 callersFunction_strip_heredoc_bodies
``text`` without its here-document bodies (see :func:`_split_heredocs`).
src/skillevaluator/tier3/eval_core/plugin_signals.py:2318
↓ 1 callersFunction_strip_spdx_capture
Drop inline comment terminators and trailing source after the SPDX value. A backtick or ``|`` ends the value only when the same delimiter opens b
src/skillevaluator/validators/license.py:46
↓ 1 callersFunction_strip_userinfo
(value: str)
src/skillevaluator/validators/claude_plugin_validate.py:138
↓ 1 callersFunction_strip_valid_frontmatter
Strip valid mapping frontmatter and return its original line offset. Raises :class:`StructuredDataLimitError` when mapping frontmatter exceeds a
src/skillevaluator/deduplication/utils/skill_collector.py:376
↓ 1 callersFunction_strongest_signal
Return the highest- (or lowest-) scoring ``name=value`` part for the signal bullet.
src/skillevaluator/evaluation/dimension_judge.py:396
↓ 1 callersFunction_structured_piece_digest
Config layout may differ across arms; its effective payload and rwx bits must match.
src/skillevaluator/tier3/loose_parts.py:1475
↓ 1 callersFunction_student_t_cdf
Student's t cumulative distribution, via the regularized incomplete beta function.
src/skillevaluator/tier3/harbor/stats.py:524
↓ 1 callersFunction_student_t_quantile
Upper quantile (probability above 0.5) of Student's t, by bisection.
src/skillevaluator/tier3/harbor/stats.py:538
↓ 1 callersMethod_submitter_usernames
Resolve the identity(ies) of whoever is submitting/validating the skill. Used (together with the skill's declared author) to decide which
src/skillevaluator/validators/security.py:3097
↓ 1 callersFunction_substitution_command_names
List what the commands in a substitution body name, in order. Every word after a command's first that names a network client, a command runne
src/skillevaluator/tier3/eval_core/checks.py:2051
↓ 1 callersFunction_subtitle
(rows: list[dict[str, Any]])
src/skillevaluator/harness/report.py:181
↓ 1 callersFunction_succeeded_conditions
()
tests/reporting/test_verdict_evidence.py:39
↓ 1 callersFunction_successful_mcp_calls
Per declared server, the MCP call outcome counts the plugin signals read from the trajectory.
src/skillevaluator/tier3/loose_parts.py:1538
↓ 1 callersMethod_suffix_ok
(self, form: str, end: int, kind: str)
src/skillevaluator/evaluation/dataset_case_removal.py:485
↓ 1 callersFunction_suggestions_for_dimensions
Default suggestions: target the weakest dimensions (SkillEvaluator parity).
src/skillevaluator/evaluation/tier3_report.py:4508
↓ 1 callersFunction_suite_system_out
Return score and threshold text when the result records them.
src/skillevaluator/reporting/junit_reporter.py:195
↓ 1 callersFunction_sum_trajectory_metric_segments
Sum independent fragments while taking the latest cumulative value per resume chain.
src/skillevaluator/tier3/harbor/collector.py:3066
↓ 1 callersFunction_summaries_are_custom_only
Return whether every condition of an agent is scored only by its custom grader. Recorded policies win; without them, no condition may declare sta
src/skillevaluator/tier3/commands.py:1774
↓ 1 callersMethod_summarize_char_names
Build a compact summary of character names in a group.
src/skillevaluator/validators/unicode_smuggle.py:492
↓ 1 callersFunction_summarize_conflict
The pooled probe pass rate, plus which probes failed in how many trials (the per-probe view).
src/skillevaluator/tier3/eval_core/plugin_signals.py:4909
↓ 1 callersFunction_summarize_coverage
Union of per-trial activation coverage. A component is ``exercised`` when it was exercised in at least one trial, ``unavailable`` when it was
src/skillevaluator/tier3/eval_core/plugin_signals.py:4973
↓ 1 callersFunction_summarize_dataset
Trim dataset cases to the question + ground truth (plus id).
src/skillevaluator/evaluation/insights_judge.py:345
↓ 1 callersFunction_summarize_mcp
(signals: Sequence[Mapping[str, Any]])
src/skillevaluator/tier3/eval_core/plugin_signals.py:4938
↓ 1 callersFunction_summarize_order
Edge totals, trials with every edge in order, and the most common edges that did not hold, with why.
src/skillevaluator/tier3/eval_core/plugin_signals.py:4878
↓ 1 callersFunction_summarize_selection
Mean precision/recall/F1 over the trials that have a number; ``decoy_call_rate`` is a share of trials.
src/skillevaluator/tier3/eval_core/plugin_signals.py:4847
↓ 1 callersMethod_summarize_skillspector_results
Add a summary success/message entry for the skillspector scan.
src/skillevaluator/validators/security.py:5763
↓ 1 callersMethod_summarize_suppressed
Return the severity counts and rule IDs of the findings a baseline suppressed.
src/skillevaluator/validators/security.py:5548
↓ 1 callersFunction_summarize_trajectory
Convert ATIF trajectory to concise text for the LLM refinement prompt.
src/skillevaluator/tier3/generate_dataset.py:1156
↓ 1 callersFunction_summarize_trials
Pick a representative slice without collapsing agents or attempts.
src/skillevaluator/evaluation/insights_judge.py:270
↓ 1 callersFunction_summary_cells
( with_value: float | None, without_value: float | None, fmt: str, delta_fmt: str, weak: f
src/skillevaluator/tier3/case_summary.py:566
↓ 1 callersFunction_summary_line
(rows: list[dict[str, Any]], prefix: str = "")
src/skillevaluator/harness/report.py:73
↓ 1 callersFunction_summary_scores
(data: dict[str, Any], *, allow_missing_status: bool = False)
src/skillevaluator/tier3/commands.py:1714
↓ 1 callersFunction_supports_descriptor_anchored_reads
Return whether this runtime can open every path component relative to a directory fd.
src/skillevaluator/deduplication/utils/skill_collector.py:88
↓ 1 callersFunction_supports_posix_catalog_io
()
src/skillevaluator/embedding/registry.py:952
↓ 1 callersFunction_tab_indent_breaks_yaml
A tab in a line's indentation that makes the frontmatter invalid YAML (a tab inside a block scalar is fine).
src/skillevaluator/harness/overlay.py:248
↓ 1 callersMethod_tail_at
(self, index: int)
src/skillevaluator/evaluation/dataset_case_removal.py:523
↓ 1 callersFunction_take_codex_item
Pop the unpaired item with this tool and arguments (and server, when known). An answer can disambiguate candidates. Conflicting identities/outcom
src/skillevaluator/tier3/plugin_trace_artifacts.py:219
↓ 1 callersFunction_take_heredoc_bodies
Pop the bodies of one command's here-documents (by delimiter, in order) off ``pending``. A ``<<`` the shell parser saw with no body (``$((1<<2))`
src/skillevaluator/tier3/eval_core/plugin_signals.py:2323
↓ 1 callersFunction_tar_members
(path: Path)
scripts/check_oss_boundary.py:992
↓ 1 callersFunction_task_case_ids
(arm_dir: Path)
src/skillevaluator/tier3/nemo_gym/runner.py:300
↓ 1 callersFunction_task_environment_context_hash
Hash the Docker context shape, modes, paths, and file contents.
src/skillevaluator/tier3/harbor/adapter.py:1640
↓ 1 callersFunction_task_resource_value
(resources: dict[str, int] | None, key: str, default: int)
src/skillevaluator/tier3/harbor/adapter.py:2206
↓ 1 callersFunction_task_stages_digest_excluded_entries
Return whether a generated task carries entries the source digest omits. Task inputs (``evals/files`` and per-entry ``files`` references) and the
src/skillevaluator/tier3/harbor/runner.py:221
↓ 1 callersFunction_task_timeout_plan
Return the largest staged agent timeout after applying Harbor scaling.
src/skillevaluator/tier3/harbor/runner.py:3547
↓ 1 callersFunction_template_negative_question
Return an author-provided off-skill question, or None when none is available.
src/skillevaluator/tier3/generate_dataset.py:393
↓ 1 callersFunction_terminate_ack_readiness_process
Kill the ACK probe and its exec-auth descendants, then reap it.
src/skillevaluator/tier3/harbor/runner.py:2475
↓ 1 callersFunction_text
(value: object)
src/skillevaluator/reporting/external_metrics.py:17
↓ 1 callersFunction_text
(fragment: str)
tests/reporting/test_tier3_unavailable_lift_labels.py:59
↓ 1 callersFunction_text_case_ids
(text: str, known_case_ids: set[str])
src/skillevaluator/evaluation/insights_judge.py:473
↓ 1 callersFunction_text_field
(item: dict, name: str, location: str)
src/skillevaluator/validators/pii_waivers.py:116
↓ 1 callersMethod_text_field
Return the first non-empty string value among ``keys``.
src/skillevaluator/validators/dependencies.py:1013
↓ 1 callersFunction_text_in_output
(text: str, output: str)
src/skillevaluator/tier3/plugin_trace_artifacts.py:214
↓ 1 callersMethod_text_list_field
Return the first non-empty list of strings among ``keys``.
src/skillevaluator/validators/dependencies.py:1022
↓ 1 callersMethod_text_position
Return the position in *text* of *position* in the masked text (a marker's start or end inside it).
src/skillevaluator/tier3/eval_core/checks.py:4604
↓ 1 callersMethod_texts
Copy the subagents and commands where Claude Code reads user agents and commands.
src/skillevaluator/tier3/loose_parts.py:1016
↓ 1 callersFunction_third_party_imports
(source: str)
tests/test_harbor_runtime_skill_isolation.py:4279
↓ 1 callersMethod_tickSize
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunction_tier1_fixes
One item per distinct critical/high finding, so a check that failed on several shows each one.
src/skillevaluator/reporting/console_ui.py:1314
↓ 1 callersFunction_tier1_results
(results: list[ValidationResult])
src/skillevaluator/reporting/benchmark.py:2134
↓ 1 callersFunction_tier1_rows
(checks: str | None, timeout: float)
src/skillevaluator/doctor.py:102
↓ 1 callersFunction_tier2_finding_is_advisory_refusal
(finding: object)
src/skillevaluator/reporting/console_ui.py:1568
↓ 1 callersFunction_tier2_finding_is_scan_failure
(finding: object)
src/skillevaluator/reporting/console_ui.py:1520
↓ 1 callersFunction_tier2_fix
A duplicate finding as a fix item; the section list collapses to ``"Title" lines a-b`` parts.
src/skillevaluator/reporting/console_ui.py:1508
↓ 1 callersFunction_tier2_result_with_host_paths
(target: Path)
tests/test_cli.py:520
↓ 1 callersFunction_tier2_results
(results: list[ValidationResult])
src/skillevaluator/reporting/benchmark.py:2138
↓ 1 callersFunction_tier2_rows
(*, verify_models: bool, timeout: float)
src/skillevaluator/doctor.py:166
↓ 1 callersFunction_tier3_chart_projection
Keep only measurements consumed by the native dimension/token charts. Derive this from the same bounded display preview as the tables. The full
src/skillevaluator/reporting/html.py:1728
↓ 1 callersFunction_tier3_final_marks
(run_dir: Path)
src/skillevaluator/reporting/console_ui.py:1553
↓ 1 callersFunction_tier3_findings
(run_dir: Path)
src/skillevaluator/reporting/console_ui.py:1562
↓ 1 callersFunction_tier3_finite_number
Return a finite display number without overflowing on huge integers.
src/skillevaluator/reporting/console_ui.py:1057
← previousnext →7,901–8,000 of 24,741, ranked by callers