MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 6 callersMethodkind
Classify ``rel`` with ``lstat`` on every component (links are never followed). A regular file with more than one hard link is ``special`` unl
src/skillevaluator/plugin_paths.py:141
↓ 6 callersMethodlast_codex_config
(self, remote_home: object)
tests/tier3/test_harbor_local_agents.py:53
↓ 6 callersMethodmatch
(self, _source)
tests/tier3/test_codex_tool_call_normalization.py:617
↓ 6 callersFunctionmetrics_for_condition
Return the standard metric contract declared by one agent condition. Recorded metrics win. Older artifacts fall back to the recorded metric set,
src/skillevaluator/tier3/harbor/report_data.py:536
↓ 6 callersFunctionnemo_gym_config_errors
Reject ``evals/config.yml`` settings the NeMo Gym backend cannot honor, before any dataset work. A setting also given on the command line is skip
src/skillevaluator/tier3/nemo_gym/backend.py:351
↓ 6 callersFunctionnormalize_agent_eval_harbor_links
Return a shallow copy with display-safe Harbor links in nested fields.
src/skillevaluator/reporting/harbor_viewer.py:179
↓ 6 callersFunctionnormalize_embedding_vector
Validate a vector and return a unit-vector copy for repeated comparisons. Zero vectors remain zero when ``allow_zero`` is enabled, matching :
src/skillevaluator/embedding/client.py:58
↓ 6 callersFunctionnormalize_token_counts
Normalize one attempt's token counters, or return ``None`` when it recorded none. ``raw`` is a report trial's ``tokens`` object (``prompt``, ``co
src/skillevaluator/tier3/token_usage.py:93
↓ 6 callersFunctionnot_applicable_list
Return the N/A-eligible metrics a stored ``not_applicable_metrics`` list names, in canonical order. A value that is not a list or tuple, and any
src/skillevaluator/tier3/harbor/metrics.py:363
↓ 6 callersFunctionobserve_wait
(monkeypatch, budget)
tests/tier3/test_budget_reservation.py:21
↓ 6 callersFunctionparse_manifest_text
Parse manifest text with the bounded parser of its format's syntax (:func:`manifest_syntax`). Raises :class:`~skillevaluator.utils.structured_dat
src/skillevaluator/plugin_formats.py:333
↓ 6 callersFunctionparse_tool_list
Parse a ``tools`` / ``allowed-tools`` value; ``None`` when the field is absent. Accepts a YAML list or a comma- or whitespace-separated string. S
src/skillevaluator/plugin_permissions.py:173
↓ 6 callersFunctionpass_at_k_summary
(pass_at_k: object, attempt_policy: object = None)
src/skillevaluator/reporting/summary_copy.py:15
↓ 6 callersMethodpaths
``value`` with local paths replaced, and each absolute path that ends in the run's directory. This machine's run, skill and home directories
src/skillevaluator/evaluation/tier3_feedback_chat.py:639
↓ 6 callersMethodpayload
(self)
src/skillevaluator/reporting/plugin_static.py:85
↓ 6 callersFunctionpermission_bypass_issues
The HIGH part of :func:`permission_flag_issues`: permission-bypass flags, and a scan that stopped early.
src/skillevaluator/validators/mcp_static.py:3359
↓ 6 callersMethodpopulate_context_post_run
(self, context: AgentContext)
src/skillevaluator/tier3/harbor/hermes_agent.py:193
↓ 6 callersFunctionproject_plugin_static
Reduce private static results once, keeping unknown and incomplete facts explicit.
src/skillevaluator/reporting/plugin_static.py:267
↓ 6 callersFunctionpublish_latest_results
Atomically replace ``latest`` with a relative symlink to ``run_id``.
src/skillevaluator/tier3/results_location.py:1055
↓ 6 callersMethodraise_if_requested
(self)
src/skillevaluator/tier3/harbor/runner.py:938
↓ 6 callersFunctionrecords
(inv, **kwargs)
tests/test_plugin_dependency_records.py:32
↓ 6 callersFunctionredact_printable_text
Strip terminal controls before masking credentials they may have split.
src/skillevaluator/utils/redaction.py:314
↓ 6 callersFunctionredact_secrets
``text`` (a command token, spec, or line, or a quoted code snippet) with its credentials removed. URL userinfo (also in ``https:user:pw@host`` an
src/skillevaluator/validators/url_policy.py:749
↓ 6 callersFunctionrefresh_finding_identities
Recompute a JSON report's finding IDs and groups from its current finding text. CI removes detected credentials from a report after it is written
src/skillevaluator/reporting/finding_summary.py:212
↓ 6 callersFunctionrender_logo
Return the inline SkillEvaluator logo for pages built outside report.html.j2. ``uid`` keeps the logo's ids unique on the page. An empty ``label``
src/skillevaluator/reporting/html.py:4090
↓ 6 callersFunctionrepository
(root: Path)
tests/test_plugin_dependency_records.py:15
↓ 6 callersFunctionrequired_check_failure_reason
Return the run-level required-check failure reason for one arm.
src/skillevaluator/tier3/attempt_verdict.py:228
↓ 6 callersFunctionresolve_policy
Resolve the active policy from CLI/env inputs. Precedence (highest to lowest): 1. ``policy_path`` overlaid on top of ``profile`` (or the defa
src/skillevaluator/validators/policy.py:344
↓ 6 callersFunctionroot
(tmp_path: Path)
tests/validators/test_private_format_boundaries.py:37
↓ 6 callersFunctionsafe
(value: object)
src/skillevaluator/tier3/result_display.py:1169
↓ 6 callersFunctionscope_skillspector_issues
Classify each issue in *issues*, in order, for Tier 1 applicability. An issue is classified by its location file and every occurrence file. I
src/skillevaluator/validators/skillspector_scope.py:480
↓ 6 callersFunctionselection_checkout
(tmp_path)
tests/ci/test_run_tier1.py:736
↓ 6 callersMethodsignal
(self)
src/skillevaluator/evaluation/tier3_report.py:191
↓ 6 callersMethodskilleval_native_command
Return the launch command and env with native setup applied (once per agent run). Raises :class:`NativeLaunchError` when a second launch-shap
src/skillevaluator/tier3/harbor/native_agents.py:158
↓ 6 callersMethodsubmit
(self, worker, job)
tests/validators/test_security.py:9441
↓ 6 callersFunctiontier1_required_checks
Return the Tier 1 checks publication requires for one content type.
src/skillevaluator/publication_evidence.py:136
↓ 6 callersFunctiontoken_usage_rows
Return the rows of ``usage`` and how many case rows the caps left out. Each agent gets an ``All cases`` row followed by its case rows; the al
src/skillevaluator/reporting/tier3_token_usage.py:214
↓ 6 callersFunctionvalidate_case_id
Return the canonical Harbor-safe form of one authored eval ID. The public Agent Skills evaluation guide uses integer IDs, so integers are int
src/skillevaluator/tier3/case_ids.py:21
↓ 6 callersFunctionvalidate_harbor_job_result
Validate Harbor's persisted aggregate trial state. Harbor returning zero is only subprocess success. A usable result must account for every
src/skillevaluator/tier3/harbor/collector.py:589
↓ 6 callersFunctionvalidate_results_root_location
Validate that a result root cannot become evaluator input or build context.
src/skillevaluator/tier3/harbor/adapter.py:776
↓ 6 callersFunctionverdict_to_severity
Map an LLM verdict to a Finding severity level.
src/skillevaluator/deduplication/intra_skill/llm_analyzer.py:226
↓ 6 callersFunctionwrite_arm
(run, arm, tokens, seconds, steps=None, score=1.0)
tests/reporting/test_resource_comparison.py:13
↓ 6 callersMethodx
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callersFunctionRe
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callersFunctionWs
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 5 callersMethod__init__
(self, category: str, status_code: int = 502)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:141
↓ 5 callersFunction_absolute_path_basename
(value: str)
src/skillevaluator/reporting/benchmark.py:2397
↓ 5 callersFunction_accepted_skill_names
(expected_skill: str | None, acceptable_skills: Any = None)
src/skillevaluator/tier3/eval_core/checks.py:3212
↓ 5 callersFunction_agent_cli_flag_issues
Agent-CLI flags in a hook, LSP, monitor, or settings config: permission bypasses, the permissive ``--permission-mode acceptEdits|auto`` (one :func
src/skillevaluator/plugin_config_risk.py:63
↓ 5 callersFunction_agent_eval_agents
(payload: dict[str, Any] | None)
src/skillevaluator/reporting/base.py:986
↓ 5 callersFunction_agent_eval_dataset_count
(payload: dict[str, Any])
src/skillevaluator/reporting/base.py:1344
↓ 5 callersFunction_agent_eval_finite_number
(value: object)
src/skillevaluator/reporting/base.py:597
↓ 5 callersFunction_agent_eval_finite_score
(value: object)
src/skillevaluator/reporting/base.py:607
↓ 5 callersFunction_agent_rates
(agent: Mapping[str, Any])
src/skillevaluator/tier3/case_summary.py:383
↓ 5 callersFunction_anthropic_body
(text, *, stop_reason="end_turn", served_by=None, model=None)
tests/tier3/test_judge_parse_robustness.py:781
↓ 5 callersFunction_arguments
(trajectory: dict[str, Any], case: dict[str, Any], **kwargs: Any)
tests/tier3/test_f90_mcp_arguments.py:39
↓ 5 callersMethod_artifact_uris
( file_paths: list[str], *, workspace_root: Path | None, scan_root: Path | Non
tests/reporting/test_reporters.py:1695
↓ 5 callersFunction_assert_requests
(client: MagicMock, *, full_body: bool, completes: bool)
tests/embedding/test_pairwise_preflight.py:50
↓ 5 callersFunction_attempt_job
(root: Path, agent: str, variant: str, case: str, attempt: int, outcome: str)
tests/tier3/test_baseline_timeouts.py:1188
↓ 5 callersFunction_attempt_ordinal
Return a carried ordinal or one unambiguous legacy suffix.
src/skillevaluator/tier3/harbor/collector.py:4694
↓ 5 callersFunction_attempt_rate
(marks: list[str])
src/skillevaluator/tier3/case_summary.py:369
↓ 5 callersFunction_average_overall
Average the pass/lift overall score across reward payloads.
src/skillevaluator/tier3/harbor/collector.py:5718
↓ 5 callersFunction_behavior_final_response_limit
Return the configured behavior final response section limit or default.
src/skillevaluator/tier3/eval_core/atif_helpers.py:241
↓ 5 callersFunction_benchmark_rows
Return the score rows (Overall and each dimension) of the Results at a Glance table. The count rows (required checks, goals met, security clean)
tests/reporting/test_tier3_incomplete_arms.py:109
↓ 5 callersFunction_bounded_reason_text
Normalize, redact, and bound evaluator explanations.
src/skillevaluator/tier3/harbor/report.py:644
↓ 5 callersMethod_build_live_table
(self, *, final: bool = False)
src/skillevaluator/tier3/harbor/progress.py:775
↓ 5 callersFunction_card_html
Return one evaluator card's markup, failing unless its ``div`` elements close.
tests/test_judge_lift_integrity.py:354
↓ 5 callersMethod_card_values
Raw credit-card values the PII scan reports for *text* written to *path*.
tests/validators/test_security.py:7924
↓ 5 callersFunction_cell
(table: str, condition: str, column: str)
tests/reporting/test_tier3_baseline_trial_views.py:185
↓ 5 callersFunction_check_baseline_skill_candidates_do_not_alias_target
( target_skill: Path, reference_skills_dir: Path | None, workspace_skill_paths: list[Path] | None,
src/skillevaluator/tier3/harbor/adapter.py:953
↓ 5 callersFunction_check_list
``value`` if it is a list of at most ``limit`` entries; otherwise the problem is recorded.
src/skillevaluator/tier3/eval_core/plugin_signals.py:641
↓ 5 callersFunction_check_string
( data: dict[str, Any], key: str, issues: list[ManifestIssue], *, required: bool = False,
src/skillevaluator/plugin_format_schema.py:83
↓ 5 callersFunction_chmod_entry
(module: object, parent: Path, mode: int)
tests/test_harbor_secure_copy.py:684
↓ 5 callersFunction_classify
(root: Path, name: str, marketplace_name: str | None = None)
tests/test_plugin_manifest.py:108
↓ 5 callersFunction_claude_code_trajectory
Build Harbor Claude Code ATIF steps for (command, output, exit code) tuples.
tests/tier3/test_error_recovery.py:930
↓ 5 callersFunction_clean_ci_env
Build an os.environ patch that clears CI vars except the given overrides.
tests/utils/test_helpers.py:166
↓ 5 callersFunction_clean_spector_result
(*, suppression_accounting: bool = False)
tests/validators/test_artifact_dir_exclusion.py:115
↓ 5 callersMethod_cleanup_bridge
(self, environment: BaseEnvironment)
src/skillevaluator/tier3/harbor/local_agents.py:892
↓ 5 callersFunction_client
()
tests/embedding/test_pairwise_preflight.py:32
↓ 5 callersFunction_cmd_executes_script
Whether a shell command invokes ``expected_script``. ``_shell`` names the interpreter running this text, when it is known: a ``-c`` payload c
src/skillevaluator/tier3/eval_core/script_invocation.py:2747
↓ 5 callersFunction_cmd_reads_skill_md
True if a shell command reads a SKILL.md via a file-view utility. Requires both a read verb (cat/sed/head/...) AND a ``SKILL.md`` filename re
src/skillevaluator/tier3/eval_core/checks.py:1765
↓ 5 callersFunction_collect
(tmp_path, agents, budget, **kwargs)
tests/tier3/test_budget_denied_arms.py:15
↓ 5 callersFunction_collect_baseline_only
(tmp_path: Path)
tests/tier3/test_skip_with_skill.py:80
↓ 5 callersFunction_collect_with_arm
(tmp_path: Path, trials: dict[str, dict[str, Any]], *, servers: list[str])
tests/tier3/test_f90_mcp_execution.py:227
↓ 5 callersFunction_collection
(root: Path)
tests/embedding/test_pairwise_preflight.py:21
↓ 5 callersFunction_compact_json
(value: object)
src/skillevaluator/reporting/html.py:1609
↓ 5 callersFunction_compact_size
(value: object)
tests/reporting/test_tier3_failure_reasons.py:856
↓ 5 callersFunction_config
(skill: Path, workspace: str)
tests/tier3/test_decoy_workspace.py:42
↓ 5 callersFunction_configure_llm
(mock_llm, provider: str = "anthropic", model: str = "claude-opus-5-5")
tests/deduplication/intra_skill/test_intra_skill_validator.py:1431
↓ 5 callersFunction_content_text
(content: Any)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1532
↓ 5 callersFunction_count
Return a non-negative whole token count; bools, strings, fractions and NaN are not counts.
src/skillevaluator/tier3/token_usage.py:88
↓ 5 callersFunction_count_derived_pass_rate_delta
Return the corrected arm-level delta derived directly from pass counts.
src/skillevaluator/tier3/harbor/collector.py:4980
↓ 5 callersFunction_create_native_skill_with_authored_retry_env
Create a native Harbor skill fixture whose task.toml authors [verifier.env] retry variables.
tests/test_harbor_task_toml_retry_env.py:205
↓ 5 callersMethod_create_pod
Disable service account token automount, block metadata server, and reject GCP-bound KSAs unless opted in.
src/skillevaluator/tier3/harbor/gke_environment.py:869
↓ 5 callersFunction_dataset_sufficiency_agent_result
( *, score: float = 0.8, paired_cases: int = 13, skill_activation: dict[str, Any] | None = Non
tests/test_tier3_progress.py:2953
↓ 5 callersMethod_declared_field
(self, field_name: str)
src/skillevaluator/plugin_discovery.py:282
↓ 5 callersFunction_descriptor_chmod_module
()
tests/test_harbor_secure_copy.py:642
↓ 5 callersFunction_detail
Bounded, printable, redacted text: terminal controls and C0/C1 characters are removed.
src/skillevaluator/tier3/mcp_proof.py:213
↓ 5 callersFunction_diagnostics
(judgments)
tests/reporting/test_diagnostic_view.py:26
↓ 5 callersFunction_dimension_agent
Return a succeeded agent with one with-skill score per named dimension.
tests/evaluation/test_tier3_benchmark_contract.py:361
← previousnext →1,601–1,700 of 24,741, ranked by callers