MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 1 callersFunction_failed_trial_case_id
Return the case of a trial the collector listed as failed, read from the trial's persisted directory.
src/skillevaluator/tier3/case_summary.py:251
↓ 1 callersFunction_failing_results
()
tests/golden/test_benchmark_card.py:260
↓ 1 callersFunction_failure_body
Return failure text with file:line when the finding records a location.
src/skillevaluator/reporting/junit_reporter.py:154
↓ 1 callersFunction_failure_diagnostic
Expose only trusted code locations and a finite set of failure labels.
ci/feedback.py:1086
↓ 1 callersFunction_failure_message
User-facing text for a failed turn; provider response bodies and key values are never echoed.
src/skillevaluator/evaluation/tier3_feedback_chat.py:1050
↓ 1 callersFunction_failure_reason
Return the real cause of a failed result for a synthetic failure case.
src/skillevaluator/reporting/junit_reporter.py:248
↓ 1 callersFunction_failure_records_by_arm
The staging's failure records (``{arm, attempt, piece, reason}``) as one bounded reason per arm. One piece that failed in several attempts is lis
src/skillevaluator/tier3/harbor/stats.py:858
↓ 1 callersFunction_failure_scan_text
The lead lines failure text is checked against: each block's first and last lines, and the result's last.
src/skillevaluator/tier3/eval_core/plugin_signals.py:1852
↓ 1 callersFunction_fake_reader
(scripts: dict[str, str | Exception])
tests/validators/test_plugin_hook_risk_bounds.py:202
↓ 1 callersFunction_fallback_action
(issue: Mapping[str, Any])
src/skillevaluator/tier3/case_summary.py:1231
↓ 1 callersFunction_fallback_manifest
(target: Path)
src/skillevaluator/publication_identity.py:536
↓ 1 callersFunction_fallback_models
(primary_model)
src/skillevaluator/tier3/harbor/templates/eval.py:922
↓ 1 callersFunction_fallback_opened_matches_named
Compare fallback metadata without assuming Windows CRT identities.
src/skillevaluator/publication_identity.py:459
↓ 1 callersFunction_fallback_suggestions
Rule-based fallback when LLM is unavailable.
src/skillevaluator/tier3/harbor/report.py:981
↓ 1 callersFunction_family
(values: Sequence[str])
src/skillevaluator/tier3/eval_core/plugin_signals.py:3669
↓ 1 callersFunction_fault_penalty
Return ``(floor, coefficient)`` for ``max(floor, 1 - coefficient * <variable>)`` in checks.py.
tests/test_metrics_reference_docs.py:151
↓ 1 callersFunction_fchmod_opened_entry
Change ``name``'s mode through a no-follow descriptor that matches ``named``. Returns False only when the entry cannot be opened for a permission
src/skillevaluator/tier3/harbor/secure_copy.py:852
↓ 1 callersFunction_feature_config
(*owners: dict[str, Any])
src/skillevaluator/reporting/dashboard_data.py:84
↓ 1 callersMethod_feed_chunk
(self, text: str)
src/skillevaluator/tier3/harbor/stream_redaction.py:215
↓ 1 callersFunction_feedback_items
Keep the readable fields of saved report feedback; evidence links are left out.
src/skillevaluator/evaluation/tier3_feedback_chat.py:923
↓ 1 callersFunction_fence_language
(info: str)
src/skillevaluator/validators/security.py:383
↓ 1 callersFunction_fetched_variables
Names of shell variables assigned downloaded content (``code=$(curl …)``).
src/skillevaluator/plugin_component_risk.py:2156
↓ 1 callersFunction_file_change_refs
(traj: dict[str, Any])
src/skillevaluator/tier3/eval_core/atif_helpers.py:743
↓ 1 callersFunction_file_exists
(path: str, roots: list[Path])
src/skillevaluator/tier3/eval_core/declarative_checks.py:152
↓ 1 callersFunction_file_field
(item: dict, location: str)
src/skillevaluator/validators/pii_waivers.py:135
↓ 1 callersFunction_filesystem_is_case_insensitive
(directory: Path)
tests/embedding/test_catalog.py:680
↓ 1 callersFunction_filled
A shell script of ``line.format(i=…)`` lines that just fits in the script size limit.
tests/validators/test_plugin_static_risk_review_fixes.py:892
↓ 1 callersMethod_filler_ok
Between a verb and its case phrase: up to three determiners, then up to three plain words.
src/skillevaluator/evaluation/dataset_case_removal.py:447
↓ 1 callersFunction_filter_rows
(rows: list[dict[str, Any]], field: str, option_rows: list[dict[str, Any]])
src/skillevaluator/reporting/dashboard_app.py:143
↓ 1 callersFunction_final_answer_reaches_judge
Return True when every character of the final answer appears in *evidence*.
src/skillevaluator/tier3/eval_core/atif_helpers.py:1147
↓ 1 callersFunction_final_response_ref
(traj: dict[str, Any])
src/skillevaluator/tier3/eval_core/atif_helpers.py:660
↓ 1 callersFunction_finalize_evaluated_source
Resolve one source identity onto every result before a report is written. Provenance is part of every report, not just the card, so it has to be
src/skillevaluator/cli.py:837
↓ 1 callersFunction_finalize_fallback_file
(path: Path, manifest: _FileManifest)
src/skillevaluator/tier3/harbor/secure_copy.py:2332
↓ 1 callersMethod_find_all_rules
Recursively find all .mdc rule files.
src/skillevaluator/validators/rules_schema.py:92
↓ 1 callersMethod_find_all_workflows
Recursively find all workflow directories.
src/skillevaluator/validators/workflows_schema.py:100
↓ 1 callersMethod_find_dependency_files
Locate dependency files in skill directory.
src/skillevaluator/validators/dependencies.py:178
↓ 1 callersFunction_find_job_dir
Find the exact Harbor job directory produced for ``job_name``.
src/skillevaluator/tier3/harbor/collector.py:325
↓ 1 callersMethod_find_proprietary_indicator
Find proprietary/restrictive license indicator in content.
src/skillevaluator/validators/license.py:629
↓ 1 callersFunction_find_script_end
(content: str, start: int)
src/skillevaluator/validators/markdown.py:94
↓ 1 callersFunction_find_toml_table_key_value_span
Return the (start, end) character span of a string value for [target_table].target_key.
src/skillevaluator/tier3/harbor/adapter.py:4925
↓ 1 callersFunction_finding
( result: ValidationResult, manifest: PluginManifestFile, check: str, message: str, *,
src/skillevaluator/plugin_dependency_records.py:38
↓ 1 callersFunction_finding
(severity: str, message: str, file_path: str, line: int)
tests/reporting/browser_navigation_fixtures.py:26
↓ 1 callersFunction_finding_case_name
Return a finding case name keyed on its location, not its position in the list. ``quality_correctness (SKILL.md:12)`` stays the same when other f
src/skillevaluator/reporting/junit_reporter.py:171
↓ 1 callersFunction_finding_field
The top-level field of the selected Claude Code manifest a SkillEvaluator finding is about, if any.
src/skillevaluator/validators/claude_plugin_validate.py:666
↓ 1 callersFunction_finding_line
(finding: Finding, private_labels: tuple[str, ...] = ())
src/skillevaluator/reporting/benchmark.py:2162
↓ 1 callersFunction_finding_occurrences
Return every location one raw finding reports. That is the finding's own location plus, when the check folded repeats of one value into this
src/skillevaluator/reporting/finding_summary.py:46
↓ 1 callersFunction_finding_rel_path
Root-relative POSIX path of a finding location (``[prefix] `` markers stripped).
src/skillevaluator/plugin_inventory.py:472
↓ 1 callersMethod_finding_to_dict
Preserve the legacy JSON null for findings without metadata.
src/skillevaluator/reporting/json_reporter.py:526
↓ 1 callersFunction_findings_result
(order: list[tuple[str, str, int | None]])
tests/reporting/test_junit_reporter.py:755
↓ 1 callersFunction_finds_at_depth
Whether the profile's grouping rules still let it find a skill ``depth`` folders below the user root ``root``. The rules read the same names as i
src/skillevaluator/harness/overlay.py:907
↓ 1 callersFunction_fingerprint
(value)
src/skillevaluator/deduplication/plugin/catalog.py:343
↓ 1 callersMethod_finish_failed_stop
Show the cursor and the stop notice when the terminal refused them after a drawing failure. Runs on the thread that closes the run, once the
src/skillevaluator/tier3/harbor/progress.py:646
↓ 1 callersFunction_finite_mean
Average non-negative values without overflowing to infinity.
src/skillevaluator/reporting/trial_charts.py:41
↓ 1 callersFunction_finite_numeric
(value: Any)
src/skillevaluator/evaluation/insights_judge.py:386
↓ 1 callersFunction_finite_score
(value: Any)
src/skillevaluator/tier3/eval_core/llm_judge.py:171
↓ 1 callersFunction_finite_score
(value)
src/skillevaluator/tier3/harbor/templates/eval.py:1183
↓ 1 callersFunction_finite_score
A finite score rounded for the chat; bools, text and non-finite values are dropped.
src/skillevaluator/evaluation/tier3_feedback_chat.py:243
↓ 1 callersFunction_first_action
Select one recorded problem, preferring unavailable scans over findings.
ci/report_views.py:446
↓ 1 callersFunction_first_error
(results: Sequence[ValidationResult])
src/skillevaluator/precommit.py:160
↓ 1 callersFunction_first_error
(result: ValidationResult)
src/skillevaluator/reporting/console_ui.py:1648
↓ 1 callersFunction_first_expected_artifact_path
(text: str)
src/skillevaluator/tier3/eval_core/atif_helpers.py:766
↓ 1 callersFunction_first_failure
Name the first Harbor exception type in an arm's errors, in words (``agent timeout error``).
src/skillevaluator/reporting/tier3_coverage.py:148
↓ 1 callersMethod_first_level_paths
Script paths a command names: quoted command strings are split once more, relative paths after ``cd <root placeholder>`` resolve against that
src/skillevaluator/plugin_component_risk.py:3232
↓ 1 callersFunction_first_line
(text: str, *, shell: bool = False)
src/skillevaluator/tier3/eval_core/plugin_signals.py:2005
↓ 1 callersFunction_first_noncoverage_reason
(checks: list[dict])
src/skillevaluator/tier3/dataset_sufficiency.py:43
↓ 1 callersFunction_first_recorded_trial_exception
Name the first trial or step exception Harbor recorded under a failed job. Harbor 0.24 records a multi-step failure, such as a verifier ``reward.
src/skillevaluator/tier3/harbor/collector.py:861
↓ 1 callersFunction_first_task_name
(dataset: Path)
src/skillevaluator/tier3/harbor/runtime_preflight.py:875
↓ 1 callersFunction_first_trial_exception_detail
Return a bounded first exception from Harbor's retained trial results.
src/skillevaluator/tier3/harbor/runtime_preflight.py:887
↓ 1 callersFunction_first_unnegated_match
A bounded excerpt of the first positive requirement, in linear scan order.
src/skillevaluator/tier3/evals_spec.py:789
↓ 1 callersFunction_first_usage
(trajectory: Mapping[str, Any] | None)
src/skillevaluator/tier3/plugin_evidence.py:155
↓ 1 callersFunction_first_use
(refs: Sequence[_Ref], calls: Sequence[_Call])
src/skillevaluator/tier3/eval_core/plugin_signals.py:4239
↓ 1 callersMethod_fitCols
(t,e,i,s)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_fitRows
(t,e,i,s)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunction_fixture_secret
Build committed fake secrets from pieces so static scanners do not flag them.
tests/tier3/test_log_jwt_redaction.py:17
↓ 1 callersMethod_flag_dependency_folder_skills
HIGH for a skill a client loads from a dependency folder inside declared skills folder ``rel_dir``. Skill discovery prunes ``node_modules/``,
src/skillevaluator/plugin_discovery.py:870
↓ 1 callersFunction_flag_duplicate_names
MEDIUM for each server name declared again by a later Claude Code source, which replaces the earlier one.
src/skillevaluator/plugin_mcp.py:416
↓ 1 callersFunction_flag_hits
Yield ``(json_path, flag, risk)`` for permission flags in a string or split across argv tokens.
src/skillevaluator/validators/mcp_static.py:3253
↓ 1 callersMethod_flag_unscanned_skill
HIGH for a skill deep in a skill's own artifact folder that Codex loads from declared folder ``rel_dir``. Skills under ``skills/`` are report
src/skillevaluator/plugin_discovery.py:859
↓ 1 callersFunction_flag_value
``True``/``False`` for a boolean flag in any common spelling, else ``None``.
src/skillevaluator/tier3/eval_core/plugin_signals.py:1894
↓ 1 callersFunction_flag_value_is_credential
Whether the value of credential flag ``flag`` is an inline credential (``--creds`` judges its password).
src/skillevaluator/validators/mcp_static.py:198
↓ 1 callersFunction_flat_value_end
Return where *value* ends when a call and a call inside a tuple are not read. A call value then ends with its first word ("HTTPBasicAuth(user," i
src/skillevaluator/tier3/eval_core/checks.py:5264
↓ 1 callersFunction_flatten_trials
Flatten per-agent trials into a single list with ``agent`` annotated.
src/skillevaluator/evaluation/tier3_report.py:3836
↓ 1 callersFunction_float_between
(value: Any, minimum: float, maximum: float, config_path: Path, field: str)
src/skillevaluator/tier3/evals_config.py:404
↓ 1 callersFunction_float_env
Read a non-negative float from the environment variable or return default.
src/skillevaluator/inference/retry.py:97
↓ 1 callersFunction_float_greater_than
( value: Any, minimum: float, config_path: Path, field: str, *, maximum: float | None
src/skillevaluator/tier3/evals_config.py:416
↓ 1 callersFunction_fmt_duration
(seconds: float | None)
src/skillevaluator/reporting/console_ui.py:109
↓ 1 callersFunction_follows_split_ansi_code
Whether ANSI codes end right before ``index`` and a line escape or path separator precedes them. So in ``/opt/a.py\\r\\x1b[K/srv/<id>`` the root
src/skillevaluator/evaluation/tier3_feedback_chat.py:437
↓ 1 callersFunction_force_kill_process_tree
Force-kill without evaluating POSIX-only signal constants on Windows.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:696
↓ 1 callersFunction_format_case_block
Render the prompt block (header + findings + trace) for a single case.
src/skillevaluator/evaluation/dimension_judge.py:128
↓ 1 callersFunction_format_case_findings
Render the per-case findings collection as prompt lines.
src/skillevaluator/evaluation/dimension_judge.py:119
↓ 1 callersFunction_format_evaluator_line
Render a single ``name: with_skill=…, baseline=…, lift=…`` prompt line.
src/skillevaluator/evaluation/dimension_judge.py:109
↓ 1 callersFunction_format_http_error_with_fallback
(error: urllib.error.HTTPError)
src/skillevaluator/tier3/eval_core/llm_judge.py:184
↓ 1 callersFunction_format_log_score
(value)
src/skillevaluator/tier3/harbor/templates/eval.py:4151
↓ 1 callersMethod_format_occurrences
Render grouped occurrence locations compactly, e.g. ``a.md lines 3, 9; b.md line 12``.
src/skillevaluator/validators/security.py:5941
↓ 1 callersFunction_format_points
Format an uplift given in displayed units; an exact zero is shown as ``±0.0 points``.
src/skillevaluator/reporting/benchmark.py:1951
↓ 1 callersFunction_format_report_number
(value: object, *, signed: bool = False)
src/skillevaluator/reporting/markdown.py:90
↓ 1 callersFunction_formatted_constant_string
(node: ast.Call, *, depth: int)
scripts/check_oss_boundary.py:257
↓ 1 callersMethod_frontmatter_author_emails
Map valid frontmatter author lines to the public contributor email.
src/skillevaluator/validators/security.py:6513
↓ 1 callersFunction_frontmatter_hook_config
The ``hooks`` block of a Markdown file's frontmatter (bounded, no-follow read), or ``None``.
src/skillevaluator/tier3/plugin_native.py:292
↓ 1 callersFunction_frontmatter_parts
The frontmatter lines and the lines after it (``None`` without a closed ``---`` block).
src/skillevaluator/tier3/plugin_native.py:679
← previousnext →6,301–6,400 of 24,741, ranked by callers