MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 2 callersFunction_bedrock_scoped_config
(session: object | None)
src/skillevaluator/tier3/harbor/runtime_preflight.py:213
↓ 2 callersFunction_behavior_expectations
(expected_behaviors, results, error="")
src/skillevaluator/tier3/harbor/templates/eval.py:3853
↓ 2 callersFunction_behavior_payload_error
Describe what is wrong with a behavior reply, or return ``None`` when it is usable. Each expected behavior is judged exactly once, identified by
src/skillevaluator/tier3/eval_core/llm_judge.py:1152
↓ 2 callersFunction_behavior_payload_error
Describe what is wrong with a behavior reply, or return ``None`` when it is usable. Each expected behavior is judged exactly once, identified by
src/skillevaluator/tier3/harbor/templates/eval.py:3870
↓ 2 callersFunction_benchmark_rows
(benchmark: str)
tests/reporting/test_activation_benchmark_html_regressions.py:55
↓ 2 callersFunction_best_elapsed
(run, traj: dict, repeats: int = 5)
tests/tier3/test_apply_patch_security.py:1192
↓ 2 callersFunction_blocking
(inventory)
tests/tier3/test_openclaw_prepare.py:290
↓ 2 callersFunction_body
(manifest, run_url, artifact_url, receipt)
ci/feedback.py:232
↓ 2 callersFunction_bottom_right_corner
(console: Any)
tests/test_tier3_progress.py:2125
↓ 2 callersFunction_bound_publication_target
Return the run's publication target when ``result.json`` belongs to this run directory.
src/skillevaluator/evaluation/tier3_feedback_chat.py:100
↓ 2 callersFunction_bound_tier1_and_tier2
(skill_name: str)
tests/reporting/test_advisory_skip_verdict.py:2350
↓ 2 callersFunction_boundary_anchors
Return exact Python comment anchors keyed by their physical line.
scripts/check_oss_boundary.py:735
↓ 2 callersFunction_bounded_detail
Census evidence or reason text of at most ``MAX_CENSUS_TEXT`` characters. Unlike names and ids (cut at the end, so the same id always reads the
src/skillevaluator/tier3/plugin_native.py:2381
↓ 2 callersFunction_bounded_join
(parts: Iterable[str], separator: str = "")
scripts/check_oss_boundary.py:249
↓ 2 callersFunction_bounded_json_text
Bound text by its ensure-ascii JSON representation, not Python characters.
src/skillevaluator/tier3/harbor/report.py:57
↓ 2 callersFunction_bounded_judge_text
Normalize trusted-shape model text before it reaches artifacts and reports.
src/skillevaluator/tier3/eval_core/llm_judge.py:163
↓ 2 callersFunction_bounded_problem
Make a rejection reason safe for logs, retry notes and failure reports. The reason quotes the judge's own text, such as an invented criterion id,
src/skillevaluator/validators/rubric_eval.py:226
↓ 2 callersFunction_bounded_report_reason
Coerce legacy/custom artifact values into bounded display text.
src/skillevaluator/tier3/harbor/report.py:433
↓ 2 callersFunction_bounded_text
(value: object, *, max_length: int)
src/skillevaluator/skillspector_provenance.py:94
↓ 2 callersFunction_bounded_tier3_preview
Return a presentation-only projection plus visible omission counts. Built failure reasons and excerpts are optional for the overview, so they
src/skillevaluator/reporting/html.py:1760
↓ 2 callersFunction_bounded_tool_observation
Keep status/error suffixes when tool output fills the observation budget.
src/skillevaluator/tier3/eval_core/log_converters.py:25
↓ 2 callersMethod_bridge_api_url
(self)
src/skillevaluator/tier3/harbor/local_agents.py:782
↓ 2 callersMethod_bridge_client_token
(self)
src/skillevaluator/tier3/harbor/local_agents.py:785
↓ 2 callersMethod_bridge_raw_exec
( self, environment: BaseEnvironment, command: str, env: dict[str, str] | None
src/skillevaluator/tier3/harbor/local_agents.py:708
↓ 2 callersMethod_bridge_server
(self)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:243
↓ 2 callersFunction_brief
(value: object, limit: int = 320)
ci/report_views.py:314
↓ 2 callersFunction_budget_left_attempts_not_run
Return budget metadata whose token cap stopped the run after its first attempt.
tests/reporting/test_tier3_completed_agent_docs.py:453
↓ 2 callersFunction_budget_number
(value: Any, *, cost: bool = False, missing: str = "not measured")
src/skillevaluator/tier3/harbor/run_budget.py:461
↓ 2 callersFunction_budget_observations
Derive reporting facts, including for legacy metadata, without changing admission.
src/skillevaluator/tier3/harbor/run_budget.py:426
↓ 2 callersMethod_build_effective_config
Extend Harbor's final user/MCP config with SkillEvaluator routing.
src/skillevaluator/tier3/harbor/local_agents.py:452
↓ 2 callersFunction_build_endpoint
Validate the permitted upstream origin and construct its only endpoint.
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:327
↓ 2 callersMethod_build_entries
( self, root: Path, content_entries: list[ContentEntry], minimum_entries: int,
src/skillevaluator/embedding/registry.py:300
↓ 2 callersFunction_build_evidence_ref_lookup
Build a lookup from each stable compact evidence key to its full dict ref. Iterates over every publishable metric and resolves its detail through
src/skillevaluator/tier3/harbor/report.py:732
↓ 2 callersFunction_build_file_manifest
(source: Path | str, allowed_root: Path | str)
src/skillevaluator/tier3/harbor/secure_copy.py:1608
↓ 2 callersFunction_build_in_pod_metadata_isolation_probe_script
Build the shell script that verifies GKE/GCE metadata endpoints are unreachable inside the pod.
src/skillevaluator/tier3/harbor/gke_environment.py:120
↓ 2 callersFunction_build_openai_response_format
Build OpenAI-compatible JSON schema response_format payload.
src/skillevaluator/inference/client.py:170
↓ 2 callersMethod_build_prompt
Build a user prompt grouping findings by file with surrounding code context.
src/skillevaluator/inference/finding_verifier.py:108
↓ 2 callersFunction_builder_reward
A failing reward whose evidence refs come from the production builder.
tests/tier3/test_suggestion_grounding.py:148
↓ 2 callersFunction_builtin_refusal_fallback_allowed
Return whether the built-in Claude Opus 4.8 refusal fallback may answer a refused judge. It is off under LLM_JUDGE_FALLBACK_POLICY=replace, and u
src/skillevaluator/tier3/harbor/templates/eval.py:992
↓ 2 callersFunction_bundle_target
``bundle_root/rel`` with its parent directories created, never through a link or outside the bundle.
src/skillevaluator/tier3/harbor/native_staging.py:198
↓ 2 callersFunction_bwrap_smoke_result
Run the probe, returning success plus bubblewrap's own failure line. The detail separates causes that otherwise share one message: ``pivot_ro
src/skillevaluator/tier3/harbor/local_sandbox.py:473
↓ 2 callersFunction_bypass_refusal
Why ``value`` must not be staged natively (a permission-bypass flag), or ``None``.
src/skillevaluator/tier3/plugin_native.py:213
↓ 2 callersFunction_calc
(expression: str)
tests/test_reference_calculator.py:17
↓ 2 callersFunction_call_matches
(refs: Sequence[_Ref], call: _Call)
src/skillevaluator/tier3/eval_core/plugin_signals.py:3496
↓ 2 callersFunction_call_with_schema_fallback
Invoke call_fn and downgrade to prompt-only on confirmed HTTP 400/422 schema errors. ``build_kwargs(include_schema)`` returns the provider reques
src/skillevaluator/inference/client.py:293
↓ 2 callersFunction_caller_home
(source: Mapping[str, str])
src/skillevaluator/validators/claude_plugin_validate.py:142
↓ 2 callersFunction_canary_captured
For each simple command of ``statement``, whether a ``$( ... )``, backtick, or ``<( ... )`` captures its output (so ``echo`` text there becomes a
src/skillevaluator/tier3/eval_core/plugin_security.py:1668
↓ 2 callersFunction_canary_expand
Read inline shell payloads (``sh -c``, ``eval``, a heredoc fed to a shell) in place of their command. A payload that is part of a pipeline, or th
src/skillevaluator/tier3/eval_core/plugin_security.py:997
↓ 2 callersFunction_canary_follow
``path`` with a leading symbolic link an earlier command made replaced by the link's target.
src/skillevaluator/tier3/eval_core/plugin_security.py:742
↓ 2 callersFunction_canary_git_subcommand
(args)
src/skillevaluator/tier3/eval_core/plugin_security.py:1219
↓ 2 callersFunction_canary_glob_matches
Shell-style path match, one component at a time; a glob matches a leading dot only literally.
src/skillevaluator/tier3/eval_core/plugin_security.py:730
↓ 2 callersFunction_canary_is_outside
(path, spec, cwd=".")
src/skillevaluator/tier3/eval_core/plugin_security.py:1419
↓ 2 callersFunction_canary_loopback_only
Every destination of the statement is a literal loopback address (``localhost``, ``127.0.0.0/8``, ``[::1]``). ``commands`` are the ``(words, name
src/skillevaluator/tier3/eval_core/plugin_security.py:1560
↓ 2 callersFunction_canary_patch_sections
``(destination, text)`` for each file an apply_patch patch writes, the path as written. A section runs from its ``Add File`` or ``Update File`` h
src/skillevaluator/tier3/eval_core/plugin_security.py:1340
↓ 2 callersFunction_canary_path_strength
``2`` when a shell word names the decoy file, the decoy directory literally, or a tainted file; ``1`` for a glob that matches only the decoy direc
src/skillevaluator/tier3/eval_core/plugin_security.py:750
↓ 2 callersFunction_canary_publishes
A package manager uploads a package (``npm publish``, ``cargo publish``, ``gem push``).
src/skillevaluator/tier3/eval_core/plugin_security.py:1472
↓ 2 callersFunction_canary_root_operands
Operands of a command that reads every file under a directory operand (a workspace root counts).
src/skillevaluator/tier3/eval_core/plugin_security.py:1647
↓ 2 callersFunction_canary_sends
A simple command runs a network client or uploads a package.
src/skillevaluator/tier3/eval_core/plugin_security.py:1478
↓ 2 callersFunction_canary_simple_commands
The word runs of one statement, split at control operators.
src/skillevaluator/tier3/eval_core/plugin_security.py:1145
↓ 2 callersFunction_canary_stdin_command
``chars`` typed into the session that runs ``session``: fed as that command's standard input (a shell session runs them as commands). Without a kn
src/skillevaluator/tier3/eval_core/plugin_security.py:1445
↓ 2 callersFunction_canary_strip_env_handoff
Drop whole-environment copies that only become a child process's ``env``.
src/skillevaluator/tier3/eval_core/plugin_security.py:1588
↓ 2 callersFunction_canary_symbolic
(args)
src/skillevaluator/tier3/eval_core/plugin_security.py:1320
↓ 2 callersFunction_canary_target_directory
The ``-t DIR`` / ``--target-directory=DIR`` of a ``cp``/``mv``/``install``/``ln`` command line.
src/skillevaluator/tier3/eval_core/plugin_security.py:1307
↓ 2 callersFunction_canary_text_words
Arguments that are text rather than file names: ``echo``/``printf`` operands (unless piped on, where a reader such as ``xargs`` may open them), a
src/skillevaluator/tier3/eval_core/plugin_security.py:1604
↓ 2 callersFunction_canary_track
Advance the compound-command nesting ``stack`` over one token; return the next ``command_position``.
src/skillevaluator/tier3/eval_core/plugin_security.py:1050
↓ 2 callersFunction_canary_units
Parts of one statement that are read on their own, as ``(tokens, isolated)`` pairs. An ``if``/``while``/``until`` test is read apart from each bo
src/skillevaluator/tier3/eval_core/plugin_security.py:1105
↓ 2 callersFunction_canary_value_or_env
The token, the expanded variable, or (``bare``, for interpreter code) the variable's bare name.
src/skillevaluator/tier3/eval_core/plugin_security.py:684
↓ 2 callersFunction_canary_write_targets
Files a simple command writes: output redirections, ``tee``, copies, ``dd of=``, ``awk``/``sed`` output, and archives a ``tar``/``zip`` command cr
src/skillevaluator/tier3/eval_core/plugin_security.py:1379
↓ 2 callersFunction_cancel_when_ready
( ready_path: Path, cancellation: runner._HarborRunCancellation, *, force_after_seconds: float
tests/test_harbor_runner_status.py:491
↓ 2 callersFunction_canonical_agent_rank_from_info
Build and rank raw Harbor agent data exactly as the canonical report does.
src/skillevaluator/evaluation/tier3_report.py:4730
↓ 2 callersFunction_canonical_dataset_json
(value: Any)
src/skillevaluator/tier3/harbor/report_data.py:100
↓ 2 callersFunction_canonical_directory_entry_name
Return the spelling recorded by the parent directory for *path*.
src/skillevaluator/publication_identity.py:657
↓ 2 callersFunction_capped_client
A client that never follows redirects, ignores proxy/.netrc settings, and caps bodies. Its TCP connections go only to ``addresses`` (the endpoint
src/skillevaluator/tier3/mcp_proof.py:419
↓ 2 callersFunction_capture_prompt
(monkeypatch, suggestion_refs)
tests/tier3/test_suggestion_grounding.py:198
↓ 2 callersFunction_capture_results_roots
(monkeypatch: pytest.MonkeyPatch)
tests/test_catalog_results.py:40
↓ 2 callersFunction_case_label
(trial: Mapping[str, Any])
src/skillevaluator/tier3/token_usage.py:154
↓ 2 callersFunction_case_matched_evidence
( case_ids: list[str], evidence_links: list[dict[str, Any]], )
src/skillevaluator/evaluation/tier3_report.py:4204
↓ 2 callersFunction_case_needs_review
(case: dict[str, Any])
src/skillevaluator/tier3/generate_dataset.py:1782
↓ 2 callersFunction_case_not_applicable
A scoreless composite result whose details carry the verifier's N/A marker.
src/skillevaluator/tier3/eval_core/checks.py:8452
↓ 2 callersFunction_case_prefix
The shared ``name-`` prefix of the case ids, shown once in the agent row.
src/skillevaluator/reporting/console_ui.py:316
↓ 2 callersFunction_catalog
(tmp_path: Path, *, skills=(), plugins=())
tests/deduplication/plugin/test_openclaw_profile.py:77
↓ 2 callersFunction_catalog_absolute_path
(catalog_path: Path)
src/skillevaluator/embedding/registry.py:945
↓ 2 callersFunction_catalog_base
The skills folder a skill is installed in: a harness root, or a workspace ``skills/`` folder (OpenClaw).
src/skillevaluator/harness/overlay.py:649
↓ 2 callersFunction_catalog_metadata_id_conflicts
Find ``metadata.id`` values shared by catalog skills. Catalog runs validate each skill on its own, so uniqueness across the catalog is checke
src/skillevaluator/cli.py:1703
↓ 2 callersFunction_catalog_results_dir
Preserve catalog-relative parents under an external Tier 3 results root.
src/skillevaluator/cli.py:1385
↓ 2 callersFunction_catalog_root_record
Keep a target's own collection relocatable; retain explicit external roots.
src/skillevaluator/tier3/decoys.py:93
↓ 2 callersFunction_catalog_skill_entry
( skill_name: str, skill_report_dir: Path, *, passed: bool, reason: str, json_report_n
src/skillevaluator/cli.py:1574
↓ 2 callersFunction_cd_target
The index of the directory a ``cd`` / ``pushd`` at ``words[index]`` changes to, past its options (``cd -P``, ``cd --``); ``len(words)`` when it na
src/skillevaluator/plugin_component_risk.py:1798
↓ 2 callersFunction_cells
(row: dict)
src/skillevaluator/tier3/history.py:368
↓ 2 callersFunction_census_row
The row for ``(hook_id, event)``, added with zero counts; ``None`` once the hook cap is reached.
src/skillevaluator/tier3/eval_core/runtime_evidence.py:62
↓ 2 callersMethod_chain_start
Start of the list of names ("a, b and c's") that ends right before ``noun``, if there is one.
src/skillevaluator/evaluation/dataset_case_removal.py:373
↓ 2 callersFunction_change
(text: str, value: float | None)
src/skillevaluator/tier3/case_summary.py:667
↓ 2 callersFunction_changelog_entry
The CHANGELOG entry that starts with these words.
tests/tier3/test_dataset_baseline_fairness.py:1042
↓ 2 callersFunction_char_node
One literal character (shared, so its positions in a name are computed once).
src/skillevaluator/plugin_component_risk.py:1241
↓ 2 callersFunction_character_column
(line: str, byte_column: int)
scripts/check_oss_boundary.py:343
↓ 2 callersFunction_chat_completion_parts
(payload: Any)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1874
↓ 2 callersFunction_chat_messages
Return an OpenAI-style message list: the optional system message, then the turns.
src/skillevaluator/inference/client.py:365
↓ 2 callersFunction_chat_response
(text: str, finish_reason: str = "stop")
tests/evaluation/test_tier3_feedback_chat.py:108
↓ 2 callersFunction_chat_usage
(payload: dict[str, Any], *, anthropic: bool)
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1371
↓ 2 callersFunction_check_author
( data: dict[str, Any], issues: list[ManifestIssue], *, label: str, allowed_keys: frozense
src/skillevaluator/plugin_format_schema.py:206
← previousnext →3,601–3,700 of 24,741, ranked by callers