MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 1 callersFunction_write_failed_job
Persist a Harbor job whose only trial failed before producing a reward.
tests/test_harbor_unscored_grading_policy.py:37
↓ 1 callersFunction_write_fake_verifier_modules
(root: Path)
tests/test_harbor_runtime_skill_isolation.py:4272
↓ 1 callersFunction_write_findings_artifact
( *, results_dir: Path, skill_name: str, agent: str, findings: list[dict[str, Any]], s
src/skillevaluator/tier3/harbor/report.py:1234
↓ 1 callersFunction_write_generated
(bundle_root: Path, rel: str, text: str)
src/skillevaluator/tier3/harbor/native_staging.py:215
↓ 1 callersFunction_write_generated_root_json
Publish one bounded generated artifact without following replacements.
src/skillevaluator/tier3/harbor/collector.py:268
↓ 1 callersFunction_write_harbor_dataset
Add one quoted task to a private Harbor dataset manifest.
src/skillevaluator/tier3/commands.py:331
↓ 1 callersFunction_write_harbor_job
Persist a Harbor 0.24 job whose trials all score 1.0, except the ``refused`` attempt.
tests/test_harbor_claude_refusal_fallback.py:272
↓ 1 callersFunction_write_harbor_job
( job_dir: Path, scores: dict[str, float], not_applicable: tuple[str, ...] = (), *, case_s
tests/test_tier3_compare.py:789
↓ 1 callersFunction_write_instruction
(task_dir: Path, question: str)
src/skillevaluator/tier3/harbor/adapter.py:1923
↓ 1 callersFunction_write_invalid_dataset
(skill: Path, count: int)
tests/reporting/test_tier3_source_validation_reporting.py:430
↓ 1 callersFunction_write_job
(jobs_dir: Path, agent: str, variant: str, trials: dict[str, dict[str, Any]])
tests/tier3/test_f90_mcp_execution.py:201
↓ 1 callersFunction_write_multistep_trial
(trial_root: Path, trajectories: list[tuple[str, dict[str, Any]]])
tests/tier3/test_trajectory_timing.py:350
↓ 1 callersFunction_write_native_job
(job: Path, *, strategy: str | None, equal_scores: bool, with_skill: bool)
tests/test_harbor_collector_judge_strategy.py:18
↓ 1 callersFunction_write_ready_file
Create the private readiness handoff without following an existing link.
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:644
↓ 1 callersFunction_write_report_checked
Atomic checked fallback for Windows and platforms without dir-fd support.
src/skillevaluator/reporting/base.py:356
↓ 1 callersFunction_write_report_posix
(output_path: Path, payload: bytes)
src/skillevaluator/reporting/base.py:253
↓ 1 callersFunction_write_result
(docs_only: bool, tier3_platform: bool)
scripts/classify_ci_changes.py:161
↓ 1 callersFunction_write_skill
(tmp_path: Path)
tests/test_harbor_secure_docker_environment.py:1589
↓ 1 callersFunction_write_skill
(tmp_path: Path, compose: dict[str, Any])
tests/test_harbor_custom_compose_safety.py:18
↓ 1 callersFunction_write_skill
(root: Path)
tests/reporting/test_skillspector_reference_summary.py:44
↓ 1 callersFunction_write_summary
(reports: Path, summary: dict)
ci/run_tier1.py:278
↓ 1 callersFunction_writes_a_file
Whether a shell call writes a file (a redirect, ``tee``, ``cp``, ``sed -i``, interpreter code...). Writes to devices such as ``/dev/null`` do not
src/skillevaluator/tier3/eval_core/plugin_signals.py:3204
↓ 1 callersMethod_wrong_file_kind
The file kind a field's paths must name when ``raw`` does not (Claude Code: ``.md`` agents, ``.json`` configs). The client's schema compares
src/skillevaluator/plugin_discovery.py:433
↓ 1 callersFunction_xml_char_allowed
Return whether *character* is legal in XML 1.0 text.
src/skillevaluator/reporting/junit_reporter.py:52
↓ 1 callersFunction_yaml_error_text
Return PyYAML's context and problem with their positions, but not the source snippet. Some errors say what is wrong only in the context, as in a
src/skillevaluator/tier3/evals_config.py:216
↓ 1 callersFunction_yaml_flow_end
Return the index just past the flow collection that opens at ``text[start]``.
src/skillevaluator/tier3/harbor/templates/eval.py:4264
↓ 1 callersFunction_yaml_flow_mapping_name
Return the string ``name`` entry of the flow mapping that opens at ``text[start]``.
src/skillevaluator/tier3/harbor/templates/eval.py:4272
↓ 1 callersFunction_yaml_node
Consume the node whose value begins at ``text[start]``. ``parent`` is the column of the key or ``-`` that owns the node; a plain or block sca
src/skillevaluator/tier3/harbor/templates/eval.py:4293
↓ 1 callersFunction_yaml_scalar_content_lines
1-indexed lines covered by YAML scalars, or None when the document cannot be parsed.
src/skillevaluator/validators/security.py:334
↓ 1 callersFunction_zero_encodes_missing_tokens
(report: dict[str, Any])
src/skillevaluator/reporting/dashboard_data.py:231
↓ 1 callersFunction_zip_members
(path: Path)
scripts/check_oss_boundary.py:950
↓ 1 callersFunctionaa
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunctionabsorb
(trials: Any, *, agent: str | None, arm: str)
src/skillevaluator/evaluation/tier3_feedback_chat.py:864
↓ 1 callersMethodacquireContext
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunctionaction
(command: list[str], identities: tuple[str, ...])
tests/test_harbor_secure_docker_environment.py:3731
↓ 1 callersFunctionadd
(scope: str, message: str)
src/skillevaluator/tier3/result_display.py:410
↓ 1 callersFunctionadd
(command: str, script: str = "run.py")
scripts/script_invocation_differential.py:300
↓ 1 callersMethodaddElements
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodaddElements
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodadd_callback
(self, callback: Callable[[], None])
src/skillevaluator/tier3/harbor/runner.py:908
↓ 1 callersFunctionadd_field
(line_number: int, line: str, *, trusted: bool)
scripts/ci/check_public_benchmarks.py:1175
↓ 1 callersMethodadd_finding
( self, finding_or_tag: Finding | str | None = None, severity: Severity | None = None,
src/skillevaluator/plugin_dependency_audit.py:66
↓ 1 callersMethodadd_multiple
Add multiple tasks (semicolon or newline separated). Clears existing tasks first.
src/skillevaluator/tier3/reference_skills/task-list/scripts/tasks.py:165
↓ 1 callersFunctionadd_probe_target
(label: str, selected_provider: ProviderConfig)
src/skillevaluator/tier3/harbor/runner.py:5828
↓ 1 callersMethodadd_task
Add a single task.
src/skillevaluator/tier3/reference_skills/task-list/scripts/tasks.py:147
↓ 1 callersMethodadjustHitBoxes
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunctionadvisory_downgraded
(summary: dict)
ci/publish_reports.py:123
↓ 1 callersMethodafterAutoSkip
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodafterBuildTicks
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodafterCalculateLabelRotation
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodafterDataLimits
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodafterFit
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodafterSetDimensions
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodafterTickToLabelConversion
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodafterUpdate
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunctionafter_outer_finishes
()
tests/test_harbor_secure_docker_environment.py:2455
↓ 1 callersFunctionagent
(passed: int, *, case_id: str)
tests/test_tier3_result_display.py:436
↓ 1 callersFunctionagent
(score: float, passed: int)
tests/test_tier3_attempt_verdict.py:561
↓ 1 callersFunctionaggregate
(reports: Path, tiers: tuple[int, ...], env: dict)
ci/run_tiers.py:689
↓ 1 callersFunctionai
(t,e,i,s)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodaliases_for
Subagent name aliases for ``arm`` (declared subagents count in the with-plugin and loose-parts arms).
src/skillevaluator/tier3/eval_core/plugin_signals.py:1403
↓ 1 callersFunctionall_env_modes
Return built-in modes followed by installed custom execution backends.
src/skillevaluator/tier3_environments.py:378
↓ 1 callersFunctionallowed_download_url
(url: str)
ci/publish_reports.py:65
↓ 1 callersFunctionan
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunctionanalyze
(text: str)
src/skillevaluator/tier3/reference_skills/text-analyzer/scripts/analyze_text.py:15
↓ 1 callersFunctionanalyze_inter_plugin
(client, user_prompt)
src/skillevaluator/deduplication/plugin/llm_analyzer.py:83
↓ 1 callersFunctionannotation_text
Escape GitHub command syntax; finding text must never create commands.
ci/run_tier1.py:283
↓ 1 callersFunctionanthropic_refusal_fallback_model
Return the server-side refusal fallback target for ``model``, or ``None`` for no fallback. ``SKILL_EVAL_LLM_REFUSAL_FALLBACK_MODEL`` replaces the
src/skillevaluator/provider_config.py:133
↓ 1 callersFunctionappend
(scope: Mapping, case: str, section: dict, runtime: Mapping | None = None)
src/skillevaluator/reporting/telemetry_view.py:176
↓ 1 callersMethodappend_raw
(self, chunk: bytes, stream: OutputStream)
src/skillevaluator/tier3/harbor/local_environment.py:193
↓ 1 callersFunctionappend_tail
(text: str)
src/skillevaluator/tier3/harbor/runner.py:3957
↓ 1 callersFunctionappend_text
(content: str)
scripts/ci/check_public_benchmarks.py:1387
↓ 1 callersMethodapplies_to
(self, server: str, host: str)
src/skillevaluator/tier3/mcp_proof.py:139
↓ 1 callersFunctionapply_native_refusals
Check each native decision against the prepared plugin snapshot. A native decision needs the snapshot, and an adapter refuses to stage a comp
src/skillevaluator/tier3/plugin_native.py:2205
↓ 1 callersMethodapps
Inventory Codex app (connector) declarations: ``apps`` or the default ``.app.json``. An app is a hosted connector that SkillEvaluator cannot
src/skillevaluator/plugin_discovery.py:1077
↓ 1 callersFunctionarguments
(pattern: re.Pattern[str])
src/skillevaluator/tier3/eval_core/checks.py:1112
↓ 1 callersFunctionarm
(name)
src/skillevaluator/tier3/history.py:357
↓ 1 callersFunctionarm
(n)
tests/reporting/test_final_render_context.py:510
↓ 1 callersFunctionarm_cost
Token and USD cost per passed case, counting every attempt. Totals are only reported when every attempt carries the counter, so a partial sum
src/skillevaluator/tier3/harbor/stats.py:1260
↓ 1 callersFunctionarm_not_applicable_metrics
Metrics that no trial of an arm scored, in metric-set order. In an arm that succeeded these are not applicable, for example the skill metrics
src/skillevaluator/tier3/harbor/stats.py:1205
↓ 1 callersFunctionarm_statuses
(condition: str)
src/skillevaluator/evaluation/tier3_report.py:4755
↓ 1 callersFunctionarm_token_efficiency
Mean per-trial token efficiency; ``None`` unless every trial has token counts.
src/skillevaluator/tier3/harbor/stats.py:1252
↓ 1 callersFunctionas
(t,e,i,s,n)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodascore
(self, **_kwargs)
tests/tier3/test_ragas_judge_evidence.py:159
↓ 1 callersFunctionassess
(*_args, **_kwargs)
tests/reporting/test_shared_run_status.py:183
↓ 1 callersFunctionat
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 1 callersFunctionattempt
(passed: bool)
tests/reporting/test_tier3_token_usage_reports.py:589
↓ 1 callersFunctionattribute_findings
Set each component's ``findings`` count from finding locations. MCP findings carry ``metadata['mcp_server']`` and are attributed to that serv
src/skillevaluator/plugin_inventory.py:498
↓ 1 callersFunctionaugment_plugin_report
Compose passive evidence with a document or CLI stream. HTML needs both snapshots before its single native template render. Other formats ret
src/skillevaluator/reporting/plugin_reports.py:81
↓ 1 callersFunctionauthor_value_problem
Return why *author* is malformed or unsafe, or ``None`` when it is usable. A usable value is either the ``Name <email@host>`` shape or plain
src/skillevaluator/validators/schema.py:84
↓ 1 callersFunctionauto_approve_issues
Find MCP-entry keys that auto-approve tool calls without a user prompt.
src/skillevaluator/validators/mcp_static.py:3484
↓ 1 callersFunctionband
(fraction: Decimal)
tests/golden/test_benchmark_card.py:1429
↓ 1 callersFunctionbar
(score: float, color: str)
src/skillevaluator/reporting/console_ui.py:281
↓ 1 callersMethodbaseline_agent_import_path
(self, agent: str, base_import_path: str | None)
src/skillevaluator/tier3/plugin_execution.py:131
↓ 1 callersFunctionbaseline_eligibility
Require a fresh, complete baseline under the current result contract.
src/skillevaluator/tier3/baseline_store.py:238
↓ 1 callersFunctionbaseline_status
Return ``partial`` when some baseline attempts scored, else ``unavailable``.
src/skillevaluator/tier3/harbor/baseline_timeouts.py:87
↓ 1 callersFunctionbaseline_task_tree_digest
Hash the staged without-skill task trees, normalizing arm-specific task names.
src/skillevaluator/tier3/baseline_store.py:108
↓ 1 callersFunctionbefore
(name, workers=workers, order=order, done=done)
tests/ci/test_run_tier1.py:1297
↓ 1 callersMethodbeforeBuildTicks
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethodbeforeCalculateLabelRotation
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
← previousnext →8,301–8,400 of 24,741, ranked by callers