MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 4 callersFunction_toml_value
Serialize the documented MCP TOML scalar, string-list, and string-dict values.
src/skillevaluator/tier3/harbor/adapter.py:2299
↓ 4 callersFunction_track_chart_constructions
(page, theme="dark")
tests/reporting/test_report_efficiency_browser.py:23
↓ 4 callersFunction_traj
()
tests/tier3/test_verified_facts.py:12
↓ 4 callersFunction_trajectory
(*calls: tuple[str, dict[str, Any], Any])
tests/tier3/test_private_loose_source.py:601
↓ 4 callersFunction_trajectory_with_failed_verification_and_long_claim
Build a trajectory with a failed command (and optional large file writes) before a 9,000-char success claim.
tests/tier3/test_behavior_evidence.py:947
↓ 4 callersFunction_tree
(root: Path, files: dict)
tests/tier3/test_plugin_dataset.py:21
↓ 4 callersFunction_trial_failure_reason
Return the failure recorded for any incomplete Harbor trial.
src/skillevaluator/tier3/harbor/collector.py:1147
↓ 4 callersFunction_trial_step_exception_details
Return ordered native multi-step exception types and display reasons.
src/skillevaluator/tier3/harbor/collector.py:845
↓ 4 callersFunction_uniform_run
Write a two-case saved run scoring 0.95 with the skill and 0.50 without it on every metric not N/A. ``with_scores`` overrides individual with-ski
tests/reporting/test_applicability_saved_runs.py:318
↓ 4 callersFunction_unscored_timeout_result
()
tests/test_tier3_result_display.py:1286
↓ 4 callersFunction_unwrap_shell_command
Skip bounded env/command/exec wrappers and return the real command index.
src/skillevaluator/tier3/eval_core/script_invocation.py:102
↓ 4 callersFunction_update
(directory, update)
tests/test_tier3_history.py:82
↓ 4 callersFunction_usage_arm
One arm of a token usage row, measured and complete.
tests/reporting/test_tier3_token_usage_reports.py:65
↓ 4 callersFunction_use_openai
(monkeypatch: pytest.MonkeyPatch)
tests/evaluation/test_tier3_feedback_chat.py:114
↓ 4 callersFunction_uses
Whether ``call`` really used something ``refs`` names: a matching identity that did not fail. A shell side channel (an MCP tool called through th
src/skillevaluator/tier3/eval_core/plugin_signals.py:4212
↓ 4 callersFunction_validate
(payload: object)
src/skillevaluator/reporting/plugin_runtime_io.py:32
↓ 4 callersFunction_validate_directory_snapshot
Require one directory identity and entry snapshot to stay stable.
src/skillevaluator/utils/secure_fs.py:255
↓ 4 callersFunction_validate_fallback_components
(path: Path, *, purpose: str, create: bool = False)
src/skillevaluator/tier3/harbor/secure_copy.py:509
↓ 4 callersFunction_validate_generated_json_value
Validate structural, numeric, and encoded-size browser safety.
src/skillevaluator/tier3/harbor/collector.py:1932
↓ 4 callersFunction_validate_opened_file
( metadata: os.stat_result, relative_path: Path, expected: os.stat_result | None, *, allow
src/skillevaluator/utils/secure_fs.py:225
↓ 4 callersFunction_validate_scalar_work
(comparison_count: int, vector_dimension: int, max_scalar_comparisons: int)
src/skillevaluator/embedding/registry.py:853
↓ 4 callersFunction_validate_staging_output_location
Reject a staging destination that could delete evaluator source data.
src/skillevaluator/tier3/harbor/adapter.py:735
↓ 4 callersFunction_validate_threshold
(threshold: float)
src/skillevaluator/embedding/registry.py:788
↓ 4 callersFunction_validate_vector
(vector: object, expected_dimension: int | None)
src/skillevaluator/embedding/registry.py:815
↓ 4 callersFunction_validate_windows_discovery_directory_snapshot
( metadata: _WindowsHandleMetadata, relative_path: Path, expected: _WindowsHandleMetadata, )
src/skillevaluator/utils/secure_fs.py:1487
↓ 4 callersFunction_validated_expected_case_ids
Validate caller-owned case identities before generated outputs are reset.
src/skillevaluator/tier3/harbor/collector.py:400
↓ 4 callersFunction_validated_native_task_steps
(task_dir: Path, config: Any)
src/skillevaluator/tier3/harbor/adapter.py:5312
↓ 4 callersFunction_validated_timeout_multiplier
Return one finite positive timeout scale accepted by every entry point.
src/skillevaluator/tier3/harbor/runner.py:1848
↓ 4 callersFunction_verify_staged_node_name
(parent_descriptor: int, node: _StagedNode)
src/skillevaluator/tier3/harbor/secure_copy.py:787
↓ 4 callersFunction_visible_text_by_class
(output: str, class_name: str)
tests/test_harbor_html_report_failures.py:46
↓ 4 callersFunction_visual_text_key
Return a confusable-aware key used only to reject visual aliases.
scripts/ci/check_public_benchmarks.py:1832
↓ 4 callersFunction_windows_open_relative_handle
Open one path component relative to a held native directory handle.
src/skillevaluator/utils/secure_fs.py:653
↓ 4 callersFunction_with_skill_overall
( data: Mapping[str, Any], metric_set: object, *, score_policy: object = DEFAULT_SCORE_POLICY )
src/skillevaluator/tier3/result_display.py:278
↓ 4 callersFunction_workspace_skills
(skill_path: Path, values: list[str | Path])
src/skillevaluator/tier3/harbor/runner.py:3403
↓ 4 callersFunction_write
(path: Path, value: object)
tests/reporting/test_failed_arm_trials.py:23
↓ 4 callersMethod_write
(self, tmp_path: Path, card: str)
tests/test_benchmark_source_provenance.py:469
↓ 4 callersFunction_write_entry_json
( task_dir: Path, entry: dict[str, Any], has_skill: bool, *, workspace_mode: str = "isolat
src/skillevaluator/tier3/harbor/adapter.py:2351
↓ 4 callersFunction_write_final_result
Project, validate, and atomically publish the final result contract.
src/skillevaluator/tier3/harbor/runner.py:668
↓ 4 callersFunction_write_manifest
(skill: Path, frontmatter: str)
tests/tier3/test_decoys.py:607
↓ 4 callersFunction_write_multistep_agent_only_result
Write a completed agent-only job whose single trial has one named step.
tests/test_harbor_runtime_preflight.py:4155
↓ 4 callersFunction_write_multistep_attempt_job
( job_dir: Path, *, root_score: float, step_scores: tuple[float, ...], failed: bool = Fals
tests/test_harbor_runner_status.py:1386
↓ 4 callersFunction_write_retained_run
(run: Path, viewer: object)
tests/reporting/test_recorded_harbor_uploads.py:129
↓ 4 callersFunction_write_separate_verifier_task
(target: Path, *, step: bool)
tests/test_harbor_runtime_skill_isolation.py:5138
↓ 4 callersFunction_write_skills
(root: Path, *names: str, line: str = f"Write to {ADDRESS}.")
tests/validators/test_pii_waivers.py:316
↓ 4 callersFunction_write_staged_entry
(run_dir: Path, task: str, entry: dict[str, object])
tests/test_harbor_runner_artifacts.py:26
↓ 4 callersFunction_written_config
(command: str | None)
tests/tier3/test_opencode_snapshot_config.py:24
↓ 4 callersFunction_written_report
The written report that lists every source error, or None when the run writes none.
tests/reporting/test_tier3_source_validation_reporting.py:52
↓ 4 callersMethodaddEventListener
(t,e,i)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 4 callersFunctionadd_evidence_links_to_suggestions
Append Harbor viewer evidence links to report suggestions when available.
src/skillevaluator/tier3/harbor/report.py:1119
↓ 4 callersFunctionagent_eval_publication_dataset_provenance
Return canonical digest provenance required by public benchmark cards.
src/skillevaluator/reporting/base.py:759
↓ 4 callersFunctionagent_eval_publication_run_id
Return the exact bounded Tier 3 run identity duplicated in its summary.
src/skillevaluator/reporting/base.py:925
↓ 4 callersFunctionagent_execution_seconds
Read Harbor's measured agent phase; never substitute total job/verifier time.
src/skillevaluator/tier3/harbor/resource_usage.py:38
↓ 4 callersFunctionagent_plugins_opt_in
Whether the bytes of a root ``plugin.json`` opt into Agent Plugins semantics. ``raw`` is the whole file. Callers do not read a file over :dat
src/skillevaluator/plugin_manifest.py:306
↓ 4 callersFunctionanalyze_codex_settings
Check project/profile approval and sandbox settings without loading Codex. Every ``[profiles.*]`` table is read (the file is already byte-bounded
src/skillevaluator/plugin_config_risk.py:338
↓ 4 callersFunctionapply_claude_init_evidence
Confirm or refute the census with Claude Code's own ``system/init`` report. A plugin skill, subagent, or command is ``loaded`` only when init lis
src/skillevaluator/tier3/plugin_native.py:2590
↓ 4 callersFunctionapprove_delegation
(monkeypatch, api)
tests/ci/test_publish_reports.py:1483
↓ 4 callersFunctionarm_observations
Use the collector's existing score and attempt rules on cached logical records.
src/skillevaluator/tier3/plugin_comparisons.py:74
↓ 4 callersFunctionartifact_entry
(name, identifier, size=100)
tests/ci/test_publish_reports.py:692
↓ 4 callersFunctionartifact_text
Defer link styling until render, keeping captured strings inert.
src/skillevaluator/reporting/terminal_styles.py:83
↓ 4 callersFunctionassert_no_writes
(api)
tests/ci/test_prepare_comments.py:61
↓ 4 callersFunctionattempt_details_hint
Return the Attempt Details section hint as one line of text.
tests/test_harbor_timeout_recovery.py:383
↓ 4 callersMethodattribute_findings
Recount static and supplied validator findings against this inventory's rows.
src/skillevaluator/plugin_inventory.py:211
↓ 4 callersFunctionb
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 4 callersFunctionbash_rule_risk
Classify one shell-tool (``Bash`` or ``PowerShell``) rule the way Claude Code classifies its allow rules. ``"unrestricted"``: the tool with no co
src/skillevaluator/plugin_permissions.py:206
↓ 4 callersFunctionbudget_fully_denied
Return whether every planned attempt was explicitly denied before execution.
src/skillevaluator/tier3/harbor/run_budget.py:470
↓ 4 callersFunctionbuild_cli_surface
Return a JSON-serializable snapshot of the CLI command tree.
tests/golden/_cli_surface.py:71
↓ 4 callersFunctionbuild_conversation_summary
Build a human-readable conversation summary for LLM judges. The summary is optimized for behavior-check and goal-accuracy prompts. Each entr
src/skillevaluator/tier3/eval_core/atif_helpers.py:161
↓ 4 callersFunctionbuilder
(*handlers)
tests/ci/test_gitlab_default_head_admission.py:61
↓ 4 callersMethodby_id
(self, name)
tests/reporting/test_report_navigation.py:50
↓ 4 callersFunctioncanary_redact
Replace the canary token in ``text``.
src/skillevaluator/tier3/eval_core/plugin_security.py:662
↓ 4 callersFunctioncanary_spec
Validate a verifier-only canary description without touching the filesystem.
src/skillevaluator/tier3/eval_core/plugin_security.py:3782
↓ 4 callersFunctioncanonical_manifest_relative
Return the supported manifest path that a root-relative path names, ignoring case for client manifests.
src/skillevaluator/plugin_manifest.py:78
↓ 4 callersMethodcapture
(args, **kwargs)
tests/validators/test_artifact_dir_exclusion.py:136
↓ 4 callersFunctioncase_insensitive
Whether ``root/alias`` (already created) is also reachable as ``root/canonical``.
tests/test_plugin_member_paths.py:35
↓ 4 callersMethodcensus_plan
What the collector needs to read this arm's load census (``mode``, declared parts, hook ids, harness).
src/skillevaluator/tier3/loose_parts.py:487
↓ 4 callersFunctioncheck_catalog_skills
C-inter compares every comparable effective skill; names never identify self.
src/skillevaluator/deduplication/plugin/catalog_checks.py:66
↓ 4 callersFunctioncheck_negative_case
Check that the agent did NOT activate the tested skill (negative case).
src/skillevaluator/tier3/eval_core/checks.py:7726
↓ 4 callersFunctionchecks
(findings)
tests/test_plugin_markdown.py:30
↓ 4 callersFunctionchecks
(result)
tests/test_plugin_member_paths.py:31
↓ 4 callersFunctionclassify_claude_dependency
Classify one Claude Code dependency as a :class:`~skillevaluator.plugin_dependencies.DependencyRow`. ``referenced`` when the plugin's marketplace
src/skillevaluator/plugin_manifest.py:740
↓ 4 callersFunctionclassify_mcp_pinning
Classify whether one MCP declaration runs an exactly-pinned package. Package runners (``npx``, ``bunx``, ``bun x``, ``pnpm dlx``, ``yarn dlx``,
src/skillevaluator/validators/mcp_static.py:2581
↓ 4 callersMethodclient
(self, service, **kwargs)
tests/test_harbor_runtime_preflight.py:1339
↓ 4 callersMethodclose
(self)
src/skillevaluator/tier3/harbor/runner.py:5366
↓ 4 callersFunctioncollect_skill_roots
Return unique skill roots for *paths*, in first-seen order.
src/skillevaluator/precommit.py:124
↓ 4 callersFunctioncollected
(content: str, offset: int = 0)
tests/deduplication/utils/test_chunker_fences.py:14
↓ 4 callersFunctioncompare_content_hashes
``content_hashes``, ``equal_across_arms``, and ``mismatches`` for the staged arms' piece hashes. A piece is a mismatch when the full and loose ar
src/skillevaluator/tier3/loose_parts.py:1490
↓ 4 callersFunctioncomplete_sample
A fixture with no limitation of its own, so each check is observable.
tests/reporting/test_private_runtime_projection.py:128
↓ 4 callersMethodcompletion
(self)
tests/test_harbor_secure_docker_environment.py:218
↓ 4 callersFunctioncomponent_finding
A ``PLUGIN_SCHEMA`` finding attributed to a plugin ``component`` (``(type, name)``) in its metadata.
src/skillevaluator/plugin_component_risk.py:558
↓ 4 callersFunctionconstrain_native_html
Reject executable markup absent from the trusted Tier 1 template.
ci/native_reports.py:82
↓ 4 callersMethodconvert
(self, value, param, ctx)
src/skillevaluator/cli.py:117
↓ 4 callersFunctioncreate_temp_config
Create a temporary configuration file for external tool configs. Caller is responsible for cleanup; consider using contextlib for scoped lifetime
src/skillevaluator/utils/tool_runner.py:491
↓ 4 callersFunctiondataset_preflight
(skill: Path)
ci/run_tiers.py:284
↓ 4 callersFunctiondecoy_source_copy_ignore
Use the agent-visible skill projection for both identity and snapshots.
src/skillevaluator/tier3/decoy_identity.py:34
↓ 4 callersMethoddescribe
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 4 callersFunctiondiagnostics
(payload)
tests/reporting/test_private_runtime_projection.py:137
↓ 4 callersFunctiondimension_lift_labels
(html, agents, best_agent)
tests/reporting/native_comparison_assertions.py:14
↓ 4 callersFunctioneffective_reasoning_effort
Return the reasoning effort SkillEvaluator sends for ``model`` on ``provider``. GPT-6 models get one on every provider except Bedrock, and Claude
src/skillevaluator/provider_config.py:191
↓ 4 callersMethodembed
(self, _texts)
tests/deduplication/plugin/test_private_dedup.py:550
↓ 4 callersMethodembed_single
Generate an embedding for a single text string.
src/skillevaluator/embedding/client.py:260
← previousnext →2,301–2,400 of 24,741, ranked by callers