MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 3 callersFunction_bh1
(report: dict)
tests/validators/test_skillspector_hook_surface.py:320
↓ 3 callersFunction_bind_publication_target
(result: ValidationResult)
tests/test_public_benchmark_gate.py:52
↓ 3 callersFunction_bind_publication_target
(result: ValidationResult, *, skill_name: str = "demo")
tests/reporting/test_reporters.py:45
↓ 3 callersMethod_block_panel
(self, block: TierBlock)
src/skillevaluator/reporting/console_ui.py:767
↓ 3 callersFunction_bot_comment
(module, identifier, marker)
tests/ci/test_feedback.py:375
↓ 3 callersFunction_bounded_console_line
Flatten one line of untrusted text and bound it for the compact view.
src/skillevaluator/reporting/console_ui.py:971
↓ 3 callersFunction_bounded_evidence_ref
Project one evidence reference into the bounded report schema.
src/skillevaluator/tier3/harbor/report.py:776
↓ 3 callersFunction_bounded_execution_error
Redact and bound one diagnostic by characters and serialized bytes.
src/skillevaluator/tier3/harbor/runner.py:701
↓ 3 callersFunction_bounded_judge_text
Normalize trusted-shape model text before it reaches artifacts and reports.
src/skillevaluator/tier3/harbor/templates/eval.py:1175
↓ 3 callersFunction_bounded_reason
Return a non-empty reason string cut to ``NOT_APPLICABLE_REASON_MAX_CHARS``, or ``None``.
src/skillevaluator/tier3/harbor/metrics.py:390
↓ 3 callersFunction_bounded_reward_metadata_text
Return one bounded, redacted collector-owned metadata label.
src/skillevaluator/tier3/harbor/collector.py:507
↓ 3 callersMethod_bridge_origin
(self)
src/skillevaluator/tier3/harbor/local_agents.py:776
↓ 3 callersMethod_broken
(self, component_type: str, declared: DeclaredPath | None, raw: Any, problem: str)
src/skillevaluator/plugin_discovery.py:277
↓ 3 callersFunction_budget
(kind)
tests/reporting/test_budget_partial_reporting.py:40
↓ 3 callersFunction_build_comparison
Build cross-agent comparison table.
src/skillevaluator/tier3/harbor/collector.py:8689
↓ 3 callersFunction_build_job_analysis_url
(job_url: str)
src/skillevaluator/evaluation/tier3_report.py:3950
↓ 3 callersFunction_build_steps
(job: dict)
tests/ci/test_image_build.py:45
↓ 3 callersFunction_by_name
(results: list[ValidationResult])
tests/test_skip_checks.py:22
↓ 3 callersFunction_calculate_nesting_depth
Calculate maximum nesting depth of control flow statements.
src/skillevaluator/validators/script_lint.py:47
↓ 3 callersMethod_call
(self, callback, *args: object)
src/skillevaluator/tier3/harbor/progress.py:994
↓ 3 callersMethod_call
(self, method: str, path: str, payload=None)
ci/github_api.py:99
↓ 3 callersFunction_call_k8s_with_timeout
Invoke a Kubernetes client method with _request_timeout, falling back when fakes omit the parameter.
src/skillevaluator/tier3/harbor/gke_environment.py:444
↓ 3 callersFunction_call_validated_json_judge
Invoke a JSON judge; on an invalid reply, retry once and name the problem. *validate* returns ``None`` for a usable reply or a short description
src/skillevaluator/tier3/eval_core/llm_judge.py:565
↓ 3 callersFunction_call_validated_json_judge
Invoke a JSON judge; on an invalid reply, retry once and name the problem. *validate* returns ``None`` for a usable reply or a short description
src/skillevaluator/tier3/harbor/templates/eval.py:3148
↓ 3 callersFunction_canary_loopback_url
A literal URL, or a ``host[:port][/path]`` operand, whose host is a loopback address.
src/skillevaluator/tier3/eval_core/plugin_security.py:1483
↓ 3 callersFunction_canary_statements
Split tokens at top-level ``;``, ``&&``, ``||``, ``&``, and newlines; groups and compound commands stay whole.
src/skillevaluator/tier3/eval_core/plugin_security.py:1074
↓ 3 callersFunction_canary_taint
(tainted, item)
src/skillevaluator/tier3/eval_core/plugin_security.py:1462
↓ 3 callersFunction_canary_tar_creates
A ``tar`` command line creates an archive (``tar czf``, ``tar -c -f``, ``--create``).
src/skillevaluator/tier3/eval_core/plugin_security.py:1301
↓ 3 callersFunction_cancel_task_repeatedly
Bound cleanup even when a coroutine suppresses its first cancellation.
src/skillevaluator/tier3/harbor/local_environment.py:276
↓ 3 callersFunction_canonical_agent_rank
Return the score/lift tuple used to rank one canonical agent payload.
src/skillevaluator/evaluation/tier3_report.py:4704
↓ 3 callersFunction_canonical_case_id
Normalize a task, trial, or entry identifier to its canonical case ID.
src/skillevaluator/tier3/harbor/collector.py:4597
↓ 3 callersFunction_canonical_text_key
Normalize layout and case without accepting lookalike characters.
scripts/ci/check_public_benchmarks.py:1827
↓ 3 callersFunction_case_comparable
(agent: Mapping[str, Any], with_case: Any, without_case: Any)
src/skillevaluator/tier3/case_summary.py:163
↓ 3 callersFunction_case_diagnostics
(case: Mapping[str, Any] | None, safe: Callable[[object], str])
src/skillevaluator/tier3/case_summary.py:480
↓ 3 callersFunction_case_ids
(agents: Mapping[str, Any])
src/skillevaluator/tier3/case_summary.py:374
↓ 3 callersFunction_case_mismatch
(agent: Mapping[str, Any])
src/skillevaluator/tier3/case_summary.py:151
↓ 3 callersFunction_cases
(payload: dict[str, Any])
tests/evaluation/test_tier3_feedback_chat.py:128
↓ 3 callersFunction_census_totals
(rows: Collection[Mapping[str, Any]])
src/skillevaluator/tier3/eval_core/runtime_evidence.py:84
↓ 3 callersFunction_changelog_entry
Return the one CHANGELOG.md bullet that starts with ``lead``, whitespace-normalized.
tests/tier3/test_baseline_timeouts.py:65
↓ 3 callersFunction_check_description
(raw: Mapping[str, Any], where: str, errors: _FieldErrors)
src/skillevaluator/tier3/eval_core/plugin_signals.py:633
↓ 3 callersFunction_check_endpoint
Apply the transport and endpoint policy; return ``(transport kind, checked addresses)``. A host name is judged by the addresses it resolves to, u
src/skillevaluator/tier3/mcp_proof.py:347
↓ 3 callersFunction_check_key
(validator_name: str)
src/skillevaluator/reporting/console_ui.py:1087
↓ 3 callersFunction_check_object
Whether ``raw`` is an object with no keys outside ``allowed_keys``; otherwise the problem is recorded.
src/skillevaluator/tier3/eval_core/plugin_signals.py:652
↓ 3 callersFunction_check_ref_alternatives
(value: Any, where: str, errors: _FieldErrors)
src/skillevaluator/tier3/eval_core/plugin_signals.py:731
↓ 3 callersFunction_check_ref_string
(value: Any, where: str, errors: _FieldErrors)
src/skillevaluator/tier3/eval_core/plugin_signals.py:673
↓ 3 callersFunction_check_regex
(value: Any, where: str, errors: _FieldErrors)
src/skillevaluator/tier3/eval_core/plugin_signals.py:814
↓ 3 callersFunction_check_string_list
( data: dict[str, Any], key: str, issues: list[ManifestIssue], *, label: str, level: I
src/skillevaluator/plugin_format_schema.py:115
↓ 3 callersFunction_check_string_or_list
( data: dict[str, Any], key: str, issues: list[ManifestIssue], *, label: str, level: I
src/skillevaluator/plugin_format_schema.py:131
↓ 3 callersFunction_checks
(passed: int)
tests/test_tier3_result_display.py:231
↓ 3 callersFunction_classify_skill_match
( observed: str, expected_skill: str, acceptable_skills: Any = None, *, fuzzy: bool = Fals
src/skillevaluator/tier3/eval_core/checks.py:3256
↓ 3 callersFunction_claude_echo_trajectory
check-18 e01: Claude passes the computed tag to release-notes as Skill arguments.
tests/tier3/test_f90_signals_handoff_prompt.py:53
↓ 3 callersFunction_claude_mcp
(tmp_path: Path, server: dict)
tests/tier3/test_private_runtime_regressions.py:50
↓ 3 callersFunction_cleanup_staged_node
(parent_descriptor: int, node: _StagedNode, *, suppress_errors: bool)
src/skillevaluator/tier3/harbor/secure_copy.py:956
↓ 3 callersFunction_clear_provider_env
(monkeypatch)
tests/test_models_command.py:32
↓ 3 callersFunction_clear_route_env
(monkeypatch: pytest.MonkeyPatch)
tests/test_harbor_claude_refusal_fallback.py:110
↓ 3 callersFunction_cli_output
(result: ValidationResult)
tests/reporting/test_tier3_incomplete_arms.py:145
↓ 3 callersFunction_cli_output
(result: Any)
tests/reporting/test_tier3_token_usage_reports.py:182
↓ 3 callersFunction_clip
(text: str, limit: int)
src/skillevaluator/tier3/eval_core/atif_helpers.py:931
↓ 3 callersFunction_clip
(value: Any, limit: int, scrub: _Scrubber)
src/skillevaluator/evaluation/tier3_feedback_chat.py:717
↓ 3 callersFunction_close_html_element_stack
Pop one exactly nested element; ambiguous markup stays fail-closed.
scripts/ci/check_public_benchmarks.py:514
↓ 3 callersFunction_code_span_value
Return what a metadata field renders inside its code span, or ``None``.
scripts/ci/check_public_benchmarks.py:251
↓ 3 callersFunction_code_text
``text`` without ``#`` comments: each line up to a ``#`` that starts a word outside quotes. A line whose quoting does not balance is kept whole,
src/skillevaluator/plugin_component_risk.py:1872
↓ 3 callersFunction_codex_fixture
()
tests/tier3/test_trajectory_timing.py:21
↓ 3 callersFunction_codex_observation_content
(item: dict[str, Any], *, include_result: bool = False)
src/skillevaluator/tier3/eval_core/log_converters.py:669
↓ 3 callersFunction_collect
(tmp_path, cases, **options)
tests/reporting/test_applicability_saved_runs.py:507
↓ 3 callersFunction_collect_file_change_evidence
Render one entry per write the agent made. Each write body and tool result is clipped to a per-entry allowance. Pass ``full=True`` for the sa
src/skillevaluator/tier3/eval_core/atif_helpers.py:443
↓ 3 callersFunction_collect_pass_reasons
Extract concise success reasons.
src/skillevaluator/tier3/harbor/report.py:577
↓ 3 callersFunction_collect_refusal
Collect a claude-code run in which the safety classifier stopped only ``refused``.
tests/test_harbor_claude_refusal_fallback.py:320
↓ 3 callersMethod_collect_streamed_output
Drain both streams within one combined hard raw-byte budget.
src/skillevaluator/tier3/harbor/local_environment.py:1323
↓ 3 callersFunction_collected
The MCP findings the plugin-root collector gives for a contained manifest's ``mcpServers`` value.
tests/validators/test_mcp_static.py:496
↓ 3 callersFunction_compact_agent_conditions
Bound condition diagnostics while preserving publication truth fields.
src/skillevaluator/evaluation/tier3_report.py:1470
↓ 3 callersFunction_compact_behavior_conversation
Keep both setup context and late outcome evidence in behavior prompts.
src/skillevaluator/tier3/eval_core/llm_judge.py:957
↓ 3 callersFunction_comparable
(agent: Mapping[str, Any])
src/skillevaluator/tier3/case_summary.py:140
↓ 3 callersMethod_compile_pii_patterns
Pre-compile all PII patterns with their merged exception lists.
src/skillevaluator/validators/security.py:6536
↓ 3 callersFunction_compiled_matcher
The parsed regex of a matcher (cached; ``_UnsupportedMatcher`` and ``_InvalidMatcher`` propagate uncached). Parse trees are immutable, so the sco
src/skillevaluator/plugin_component_risk.py:1421
↓ 3 callersFunction_complete_mean
Average values only when every expected constituent is finite.
src/skillevaluator/evaluation/tier3_report.py:5123
↓ 3 callersFunction_completeness
(**kwargs: Any)
tests/tier3/test_private_ladder_stats.py:502
↓ 3 callersFunction_component_names
Casefolded declared skill, subagent, and command names, bare and under the plugin namespace. The subagents and commands an arm does not declare (
src/skillevaluator/tier3/eval_core/plugin_signals.py:3582
↓ 3 callersFunction_compose_client_credential_values
Return credential-bearing values used by the Docker Compose client.
src/skillevaluator/tier3/harbor/secure_docker_environment.py:273
↓ 3 callersFunction_condition
(agent: Mapping[str, Any], condition: str)
src/skillevaluator/tier3/token_usage.py:261
↓ 3 callersFunction_condition_quality_available
Return whether a condition may contribute score-bearing report fields.
src/skillevaluator/evaluation/tier3_report.py:2053
↓ 3 callersFunction_config_separator_name_ends
Return where each config set NAME in *text* ends that names a credential and is followed by a separator word.
src/skillevaluator/tier3/eval_core/checks.py:4964
↓ 3 callersMethod_contain_main_and_reap_compose
( self, process: asyncio.subprocess.Process, communication: asyncio.Task[Any],
src/skillevaluator/tier3/harbor/secure_docker_environment.py:1880
↓ 3 callersFunction_contains_any_term
(text: str, terms: Iterable[str])
src/skillevaluator/validators/quality_score.py:183
↓ 3 callersFunction_content_parts
Bounded text of each content block (one part for a plain string/object).
src/skillevaluator/tier3/eval_core/plugin_signals.py:1766
↓ 3 callersFunction_context
(monkeypatch, payload)
tests/reporting/test_native_overview_comparison.py:59
↓ 3 callersMethod_context_for_staging
(self)
src/skillevaluator/tier3/plugin_evidence.py:377
↓ 3 callersMethod_convert_hermes_session_to_atif
(self, jsonl_text: str, session_id: str)
src/skillevaluator/tier3/harbor/hermes_agent.py:131
↓ 3 callersFunction_copied_context_prefix_length
Return the leading copied-context count, rejecting non-prefix markers.
src/skillevaluator/tier3/harbor/collector.py:1854
↓ 3 callersFunction_copy_selected_environment
(environ: Mapping[str, str], names: Iterable[str])
src/skillevaluator/validators/security.py:638
↓ 3 callersFunction_count
(value: object)
src/skillevaluator/tier3/decoys.py:162
↓ 3 callersFunction_count
(value: object)
src/skillevaluator/tier3/evidence_volume.py:9
↓ 3 callersMethod_count_tokens
(self, count: int)
src/skillevaluator/utils/structured_data.py:311
↓ 3 callersFunction_coverage
(arm: object)
src/skillevaluator/reporting/tier3_token_usage.py:89
↓ 3 callersMethod_create_gitleaks_config
Create Gitleaks config with NVIDIA rules and allowlists. Generated output dirs (results/, versions/ and their dot forms) hold Tier 3
src/skillevaluator/validators/secrets.py:190
↓ 3 callersMethod_create_minimal_workflow
Helper to create minimal workflow structure.
tests/validators/test_workflows_schema.py:397
↓ 3 callersFunction_create_step_entries
(trial_dir: Path, names: list[str], *, directories: bool)
tests/test_issue55_collector_truth.py:2200
↓ 3 callersFunction_cursor
(root: Path, files: dict)
tests/validators/test_plugin_static_risk_review_fixes.py:46
↓ 3 callersFunction_custom_metric_value
Read one custom metric without materializing every custom metric in a reward.
src/skillevaluator/evaluation/tier3_report.py:2944
↓ 3 callersFunction_dataset_entries
(run_dir: Path)
src/skillevaluator/tier3/case_summary.py:1319
← previousnext →2,601–2,700 of 24,741, ranked by callers