MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 7 callersFunction_write_json
(path: Path, payload: dict)
tests/test_judge_lift_integrity.py:32
↓ 7 callersFunction_write_json
(path: Path, value: object)
tests/test_harbor_runner_artifacts.py:113
↓ 7 callersFunction_write_json
(path: Path, data: Any)
tests/tier3/test_baseline_timeouts.py:84
↓ 7 callersFunction_write_skill
(root: Path, name: str, author_line: str | None)
tests/golden/test_policy_behavior.py:31
↓ 7 callersFunction_write_skill_with_id
(root: Path, name: str, author_line: str, metadata_id: str)
tests/golden/test_policy_behavior.py:108
↓ 7 callersFunctionagent_eval_dimension_verdict
Recompute a Tier 3 verdict from complete supported-agent dimensions.
src/skillevaluator/reporting/base.py:1316
↓ 7 callersMethodapply
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 7 callersFunctionassess_tier3_evidence
Classify Tier 3 evidence once for every reporter.
src/skillevaluator/reporting/base.py:2156
↓ 7 callersFunctionattempt_details_rows
Return each Attempt Details body row as its visible text.
tests/test_harbor_timeout_recovery.py:377
↓ 7 callersFunctionaverage_custom_metrics
Average custom metrics across rewards inside the publication envelope.
src/skillevaluator/tier3/harbor/metrics.py:792
↓ 7 callersFunctionchunk_python
Extract docstrings and function/class signatures via AST.
src/skillevaluator/deduplication/utils/chunker.py:250
↓ 7 callersFunctionclassify_plugin_dependencies
Classify every ``skills.refs`` / ``rules.refs`` entry of a manifest mapping. ``bundled_skills`` are plugin-root-relative ids (``skills/<name>``)
src/skillevaluator/plugin_dependencies.py:815
↓ 7 callersFunctionclient
(cls, **opener)
tests/ci/test_github_api.py:46
↓ 7 callersMethodcollect_evidence
(self, jobs_dir: Path, result: dict[str, Any] | None)
src/skillevaluator/tier3/plugin_execution.py:182
↓ 7 callersFunctioncompute_dimensions_deterministic
Compute dimensions from raw evaluator scores without LLM. Uses weighted averages per the SADD Section 5.2 mapping. Also computes dimension-le
src/skillevaluator/evaluation/dimension_judge.py:299
↓ 7 callersMethodconfigure
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 7 callersFunctioncontained_directory_alias_target
Return the in-root target that an unselected directory alias names, or ``None``. ``relative`` is the alias path below the scan root. Only the raw
src/skillevaluator/utils/tier2_paths.py:67
↓ 7 callersMethodcreate_dataset
Create or preview a synthetic dataset and return its outcome.
src/skillevaluator/evaluation/service.py:75
↓ 7 callersMethoddetect_skill_type
Auto-detect skill type from directory structure. Returns one of: script-based, lib-based, resource-based, guide-only, hybrid.
src/skillevaluator/validators/quality_score.py:300
↓ 7 callersFunctioneffective_chunk_limit
Return the most chunks whose all-pairs comparison fits the scalar-work budget.
src/skillevaluator/deduplication/intra_skill/intra_skill_validator.py:65
↓ 7 callersMethodemit
(self, event: ProgressEvent)
src/skillevaluator/tier3/harbor/progress.py:249
↓ 7 callersMethodemit
(self, event: ProgressEvent)
src/skillevaluator/tier3/harbor/runner.py:5347
↓ 7 callersMethodengine_kwargs
(self)
src/skillevaluator/evaluation/options.py:94
↓ 7 callersMethoderrors
(self, source: str, raw: object)
src/skillevaluator/reporting/plugin_runtime.py:102
↓ 7 callersFunctionevaluate_plugin
Apply a profile's plugin rules; components the harness does not deliver become findings.
src/skillevaluator/harness/plugin.py:124
↓ 7 callersFunctionevidence_volume_for_conditions
( conditions: object, *, trial_failures: object = None, baseline_only: bool = False, datas
src/skillevaluator/tier3/evidence_volume.py:52
↓ 7 callersFunctionexisting_scan
(repository, monkeypatch)
tests/test_ci_onboarding_writes.py:16
↓ 7 callersFunctionfix_first_for_results
Return the Tier 1 "Fix first" entries exactly as the report for *results* lists them. The CI index uses this so its "Needs attention" line names
src/skillevaluator/reporting/html.py:4069
↓ 7 callersMethodfor_run
(cls, run_dir: Path, *, skill_dir: Path | None = None, home: Path | None = None)
src/skillevaluator/evaluation/tier3_feedback_chat.py:604
↓ 7 callersFunctionformat_character_facts
Describe non-ASCII whitespace and Cf characters without changing the text. Positions count Unicode code points, starting at one; only LF advances
src/skillevaluator/tier3/eval_core/character_facts.py:17
↓ 7 callersMethodgetContext
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 7 callersMethodgetMatchingVisibleMetas
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 7 callersMethodgetProps
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 7 callersMethodgetSortedVisibleDatasetMetas
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 7 callersMethodgraphql
Read server-owned relationships at this instance's fixed GraphQL endpoint.
ci/feedback.py:631
↓ 7 callersMethodisDatasetVisible
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 7 callersFunctionis_gke_workload_identity_allowed
Return True only when the operator explicitly opts into GKE Workload Identity.
src/skillevaluator/tier3/harbor/gke_environment.py:366
↓ 7 callersFunctionload_trajectory_with_fallback
Load ``trajectory.json`` if present and non-empty; else reconstruct from agent logs. Prefers ``claude-code.txt`` (structured JSONL) over ``cursor
src/skillevaluator/tier3/eval_core/log_converters.py:1005
↓ 7 callersFunctionlogical_trial_reward_groups
Group persisted reward rows by collector-issued logical trial identity. Older or independently produced artifacts do not carry the opaque identit
src/skillevaluator/tier3/harbor/report_data.py:582
↓ 7 callersMethodmatches
Whether the matcher matches anywhere in ``subject``, like JavaScript's ``RegExp.test``.
src/skillevaluator/plugin_component_risk.py:1460
↓ 7 callersFunctionnormalize_expected_behaviors
(value: Any)
src/skillevaluator/tier3/eval_core/optional_reference.py:24
↓ 7 callersFunctionnormalize_harbor_viewer_for_display
Return display-safe Harbor links from a canonical Tier 3 payload.
src/skillevaluator/reporting/harbor_viewer.py:12
↓ 7 callersFunctionnormalized_local_path
Normalize a local destination, optionally rejecting anchors acquired in normalization. Explicit URL schemes and URLs whose raw path begins with `
src/skillevaluator/validators/markdown.py:63
↓ 7 callersFunctionordered_run_directories
Return result directories in shared newest-first order.
src/skillevaluator/tier3/results_location.py:1004
↓ 7 callersFunctionparallel_cli
Fake CLI for concurrent runs; ``before(name)`` runs inside the call.
tests/ci/test_run_tier1.py:1253
↓ 7 callersMethodparse_for_audit
What a client reads from this manifest, for checks that must see every component (``None`` if unusable). The bounded strict read comes first.
src/skillevaluator/plugin_manifest.py:220
↓ 7 callersFunctionparse_nested_model
Parse a dict into a nested Pydantic model. Common pattern for parsing metadata dicts into their respective models. Args: model_class
src/skillevaluator/models/field_validators.py:34
↓ 7 callersMethodquery
Compare a single text against all indexed entries. Useful for checking a new item against the existing registry.
src/skillevaluator/embedding/registry.py:409
↓ 7 callersMethodrecalculate_from_findings
Rebuild passed, errors, warnings, and summary counts from current findings. Call after mutating finding severities (e.g. LLM verification dow
src/skillevaluator/models/result.py:566
↓ 7 callersFunctionrecord
(hook="hook", event="PreToolUse", code=0, duration=5)
tests/tier3/test_private_runtime_evidence.py:16
↓ 7 callersFunctionrender_help
Render a command's help screen as a (optionally colorized) string.
src/skillevaluator/cli_help.py:289
↓ 7 callersMethodreply
Answer one question; a failed or interrupted turn leaves the history unchanged.
src/skillevaluator/evaluation/tier3_feedback_chat.py:1032
↓ 7 callersFunctionresolve_agents
Resolve an explicit agent list or choose the provider's native default.
src/skillevaluator/tier3/commands.py:379
↓ 7 callersFunctionresolve_git_remote_url
Resolve a local path to a browsable HTTPS URL if inside a git repo. Detects the git remote origin, converts SSH/HTTPS URLs to a browsable HTT
src/skillevaluator/utils/helpers.py:163
↓ 7 callersFunctionresolve_retry_config
Resolve retry and backoff limits from environment variables with safe defaults.
src/skillevaluator/inference/retry.py:109
↓ 7 callersFunctionresolve_skill_path_for_run
Return the skill path used to build the run's payload. An explicit ``skill_path`` must name the skill the run recorded, so a different direct
src/skillevaluator/evaluation/tier3_feedback_chat.py:142
↓ 7 callersFunctionresults
(tier, name="demo", *, digest="a", verdict="pass")
tests/ci/test_benchmark_reports.py:17
↓ 7 callersFunctionroot_help
(width=80, *, color=False)
tests/test_cli_banner.py:31
↓ 7 callersMethodrun
(argv, **kwargs)
tests/utils/test_tool_runner.py:553
↓ 7 callersFunctionrun_admission
( tmp_path, monkeypatch, stage, *, permission="write", event_name="pull_request",
tests/ci/test_github_admission_workflow.py:18
↓ 7 callersFunctionrun_directory_sort_key
Return shared ordering metadata for one result directory. Exact-timestamp historical runs retain the prior result or pre-result summary contr
src/skillevaluator/tier3/results_location.py:912
↓ 7 callersFunctionsafe_llm_labels
Keep configured identifiers readable without emitting URLs or control text.
src/skillevaluator/inference/diagnostics.py:40
↓ 7 callersFunctionsafe_name
A filesystem- and shell-safe token for generated file names and census rows.
src/skillevaluator/tier3/plugin_native.py:207
↓ 7 callersFunctionsafe_url
(value: object)
src/skillevaluator/reporting/harbor_viewer.py:241
↓ 7 callersFunctionsanitize_generation_record
(value: object)
src/skillevaluator/tier3/dataset_provenance.py:11
↓ 7 callersFunctionsanitize_tier3_console_text
Project untrusted Tier 3 text into one bounded terminal-safe line.
src/skillevaluator/reporting/console_ui.py:1029
↓ 7 callersFunctionsave_review
(review, doc)
tests/test_trace_import.py:50
↓ 7 callersFunctionscore_to_grade
Convert numeric score to letter grade.
src/skillevaluator/models/quality.py:80
↓ 7 callersMethodset_handler
Register a per-request transport handler.
tests/test_retry.py:420
↓ 7 callersFunctionskillspector_file_role
Classify a scan-relative location by its role in an agent artifact.
src/skillevaluator/validators/skillspector_scope.py:331
↓ 7 callersMethodsleep
(self, seconds)
tests/tier3/test_judge_policies.py:38
↓ 7 callersMethodstaged_skills
Every skill the staged plugin loads: ``(census name, plugin-relative dir, copy source or None)``. Claude Code reads ``skills/<name>/SKILL.md`
src/skillevaluator/tier3/plugin_native.py:1448
↓ 7 callersMethodsubmit
(self, worker, job)
tests/test_catalog_results.py:34
↓ 7 callersMethodtally_severity
Increment the per-severity counter for a finding. INFO and any non-:class:`Severity` value are a no-op, matching the historical behav
src/skillevaluator/models/result.py:315
↓ 7 callersFunctiontier3_case_rows
Tier 1-style case rows: one ✓/✗ per test case and condition, plus passed/failed counts.
src/skillevaluator/reporting/console_ui.py:350
↓ 7 callersMethodtier_done
(self, index: int, *, failed: bool = False, rows: list[TierRow] | None = None)
src/skillevaluator/reporting/console_ui.py:624
↓ 7 callersFunctiontoml_quote
Serialize *value* as a TOML basic string.
src/skillevaluator/tier3/toml_utils.py:12
↓ 7 callersFunctiontree
(*, external=False)
tests/tier3/test_plugin_evidence_review.py:28
↓ 7 callersFunctiontrial
(entry="case", tokens=100, seconds=2)
tests/tier3/test_resource_usage.py:13
↓ 7 callersFunctionunverified_case_ids_note
Explain the ``(unverified case ID)`` mark, or name the cases *rows* leaves out. The mark lives on a case row, so its explanation is only useful o
src/skillevaluator/reporting/tier3_token_usage.py:154
↓ 7 callersFunctionurl_scheme
The lower-case scheme ``text`` starts with (``https`` for ``HTTPS://h/``), or ``None`` when it has none.
src/skillevaluator/validators/url_policy.py:554
↓ 7 callersMethodvalidate
Validate workflow(s) at path for schema and structure compliance.
src/skillevaluator/validators/workflows_schema.py:63
↓ 7 callersFunctionwi
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 7 callersFunctionwithout_scripts
Remove complete parsed script nodes, preserving all other markup and attributes.
tests/reporting/report_markup_assertions.py:40
↓ 7 callersMethodwrap
( self, argv: list[str], *, workdir: Path, write_roots: list[Path] | t
src/skillevaluator/tier3/harbor/local_sandbox.py:182
↓ 7 callersFunctionwrapper_skill_name
(plugin_name: str)
src/skillevaluator/tier3/plugin_prepare.py:575
↓ 7 callersFunctionwrite_export
(path, sessions)
tests/test_trace_import.py:37
↓ 6 callersFunctionBa
(t,e,i,s)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 6 callersFunctionCi
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 6 callersFunctionG
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 6 callersFunctionM
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 6 callersFunctionOi
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 6 callersFunctionS
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 6 callersFunction_agent_display_label
( name: str, agent: dict[str, Any], private_labels: tuple[str, ...] = (), )
src/skillevaluator/reporting/benchmark.py:1464
↓ 6 callersFunction_agent_eval_consistent_text_field
Return a duplicated text field only when every persisted value agrees.
src/skillevaluator/reporting/base.py:618
↓ 6 callersFunction_agent_runtime_failure_reason
Return why a trial cannot produce a valid score.
src/skillevaluator/tier3/harbor/collector.py:926
↓ 6 callersFunction_arm_key
(arm: str)
src/skillevaluator/tier3/loose_parts.py:677
↓ 6 callersFunction_artifact_expectation_reward
(entry_id="case-a")
tests/tier3/test_suggestion_grounding.py:171
↓ 6 callersFunction_assert_no_fragment
(value: str | bytes, secret: str)
tests/ci/test_secret_redaction.py:614
↓ 6 callersFunction_assert_process_exits
Assert that a process signalled by cleanup leaves the process table. Cleanup returns once its group probe finds no live member. On macOS that
tests/test_harbor_local_mode.py:165
← previousnext →1,301–1,400 of 24,741, ranked by callers