MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 1 callersFunction_build_invocation
(results: list[ValidationResult])
src/skillevaluator/reporting/sarif_reporter.py:261
↓ 1 callersMethod_build_issue_message
Build a display message from skillspector issue fields.
src/skillevaluator/validators/security.py:5754
↓ 1 callersMethod_build_issue_metadata
Extract optional confidence and intent into a metadata dict.
src/skillevaluator/validators/security.py:5742
↓ 1 callersMethod_build_lookup_tables
Pre-compute lookup structures from YAML config for fast scanning.
src/skillevaluator/validators/unicode_smuggle.py:76
↓ 1 callersFunction_build_metadata_probe_container
Build the trusted companion container used to verify GKE metadata isolation in the pod network namespace.
src/skillevaluator/tier3/harbor/gke_environment.py:242
↓ 1 callersFunction_build_metadata_probe_init_container
Build the trusted init container that verifies GKE metadata isolation before any application container starts.
src/skillevaluator/tier3/harbor/gke_environment.py:230
↓ 1 callersFunction_build_openai_response_format
(schema, schema_name="judge_response")
src/skillevaluator/tier3/harbor/templates/eval.py:1414
↓ 1 callersFunction_build_provenance
Assemble the Diagnostics ``provenance`` block. Mirrors SkillEvaluator's Harbor provenance: per-agent raw evaluator scores and lift feed the "
src/skillevaluator/evaluation/tier3_report.py:1362
↓ 1 callersMethod_build_reference_string
Build CWE/OWASP reference string from metadata.
src/skillevaluator/validators/code_risk.py:504
↓ 1 callersFunction_build_tree_manifest_fallback
( source: Path, allowed_root: Path, *, role: str, ignore: IgnoreCallback | None, )
src/skillevaluator/tier3/harbor/secure_copy.py:619
↓ 1 callersFunction_build_tree_manifest_posix
( source: Path, allowed_root: Path, *, role: str, ignore: IgnoreCallback | None, )
src/skillevaluator/tier3/harbor/secure_copy.py:472
↓ 1 callersFunction_build_verified_facts_section
Build the VERIFIED FACTS header string to prepend to prompt_evidence.
src/skillevaluator/tier3/eval_core/atif_helpers.py:1103
↓ 1 callersFunction_build_vertex_openapi_adc_config
Build a ProviderConfig using Google Application Default Credentials for Vertex OpenAPI.
src/skillevaluator/provider_config.py:507
↓ 1 callersFunction_build_wheel
Write a standards-conforming pure-Python wheel for the demo backend.
tests/test_custom_backend_package.py:105
↓ 1 callersFunction_built_texts
Return texts that tests build from their parameters, as the masker reads them.
tests/tier3/test_security_finding_evidence.py:4398
↓ 1 callersFunction_builtin_agent
Whether a bare subagent name reaches ``agent``'s own built-in agent, though a declared plugin agent has it.
src/skillevaluator/tier3/eval_core/plugin_signals.py:2829
↓ 1 callersFunction_bundle_budgets
Return effective bundle budgets taking into account runtime overrides.
src/skillevaluator/tier3/eval_core/atif_helpers.py:922
↓ 1 callersFunction_bundle_facts
(skill_path: Path, manifest: Path)
src/skillevaluator/harness/overlay.py:304
↓ 1 callersFunction_bundle_files
(skill_path: Path, manifest: Path)
src/skillevaluator/harness/overlay.py:274
↓ 1 callersFunction_bundled_hint
(name: str | None, bundled_by_leaf: Mapping[str, str])
src/skillevaluator/plugin_dependencies.py:598
↓ 1 callersFunction_bwrap_argv
Build the bubblewrap launcher argv. Read-only system view, the run directories as the only writable binds, no inherited privileges, loopback-
src/skillevaluator/tier3/harbor/local_sandbox.py:514
↓ 1 callersFunction_bwrap_smoke_argv
Argv proving bwrap can create a namespace AND exec a real binary. Binds the exec-critical system dirs, not just /usr: ``/usr/bin/true`` is dy
src/skillevaluator/tier3/harbor/local_sandbox.py:442
↓ 1 callersMethod_cache_init_events
Read loose-arm startup evidence once, alongside the observer's artifact cache.
src/skillevaluator/tier3/plugin_comparisons.py:292
↓ 1 callersMethod_cachedScopes
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_calculateBarIndexPixels
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_calculateBarValuePixels
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersMethod_calculatePadding
(t,e,i,s)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunction_calculate_jitter_delay
Calculate exponential backoff with full jitter.
src/skillevaluator/tier3/harbor/templates/eval.py:1754
↓ 1 callersFunction_call
(command: str, workdir: str | None = None)
tests/tier3/test_issue192_security_false_positives.py:52
↓ 1 callersFunction_call_anthropic
Call the Messages API; ``provenance["model"]`` becomes the model that served the reply.
src/skillevaluator/tier3/harbor/templates/eval.py:1962
↓ 1 callersFunction_call_bedrock
Call Bedrock Converse; ``provenance["model"]`` becomes the model that served the reply.
src/skillevaluator/tier3/harbor/templates/eval.py:2120
↓ 1 callersFunction_call_cwd
(args: Mapping[str, Any], step_cwd: str | None, root_cwd: str | None)
src/skillevaluator/tier3/eval_core/plugin_signals.py:3281
↓ 1 callersFunction_call_input_schema
``(label, schema)`` when the server this MCP call went to published an input schema for its tool.
src/skillevaluator/tier3/eval_core/plugin_signals.py:4030
↓ 1 callersFunction_call_judge_model
Send one judge prompt to *model*; ``provenance["model"]`` becomes the model that answered.
src/skillevaluator/tier3/harbor/templates/eval.py:2315
↓ 1 callersFunction_call_openai_compatible
( prompt, provider, model, max_tokens, temperature, response_schema=None, schema_name="judge_response" )
src/skillevaluator/tier3/harbor/templates/eval.py:2223
↓ 1 callersFunction_call_public_llm_with_provenance
( prompt, model=None, max_tokens=4096, temperature=0.0, allow_model_fallback=True, res
src/skillevaluator/tier3/harbor/templates/eval.py:2417
↓ 1 callersFunction_call_reads
Whether a call that did not fail read ``artifact`` (resolved). An MCP tool reads it when the tool is not a write or delete tool and the path
src/skillevaluator/tier3/eval_core/plugin_signals.py:4424
↓ 1 callersFunction_call_required_judge
Run a required LLM judge under a bounded wall-time deadline and normalize its result. The result records the model that produced it (``judge_mode
src/skillevaluator/tier3/harbor/templates/eval.py:3966
↓ 1 callersFunction_call_status
(result: Any, statuses: Mapping[str, str])
src/skillevaluator/tier3/plugin_trace_artifacts.py:370
↓ 1 callersFunction_call_strict_judge
Ask the judge under LLM_JUDGE_RETRY_POLICY=strict. At most _STRICT_JUDGE_MAX_MODELS candidates are tried, the primary first, with at most _ST
src/skillevaluator/tier3/harbor/templates/eval.py:2371
↓ 1 callersFunction_call_writes
Whether a call that did not fail wrote ``artifact`` (resolved).
src/skillevaluator/tier3/eval_core/plugin_signals.py:4395
↓ 1 callersFunction_caller_blob
(api, repository: str, sha: str, *, delegate="")
ci/publish_reports.py:523
↓ 1 callersFunction_calls_detail
(counts: Mapping[str, int])
src/skillevaluator/tier3/mcp_proof.py:882
↓ 1 callersFunction_can_restore_custom_metric_name
Allow safe names plus narrow, explicitly metric-shaped secret terms.
src/skillevaluator/tier3/harbor/collector.py:6435
↓ 1 callersMethod_can_safely_open_under_skill
Return whether *file_path* resolves to a regular file inside the skill.
src/skillevaluator/inference/finding_verifier.py:89
↓ 1 callersFunction_canary_args_text
(args)
src/skillevaluator/tier3/eval_core/plugin_security.py:1454
↓ 1 callersFunction_canary_assigned
Variables a simple command assigns: prefix assignments, ``export``-style builtins, ``read``, ``for``.
src/skillevaluator/tier3/eval_core/plugin_security.py:1205
↓ 1 callersFunction_canary_client_loopback
Every destination a ``curl``/``wget``/HTTPie command line can reach is a literal loopback address. No proxy, ``--connect-to``, ``--resolve``, con
src/skillevaluator/tier3/eval_core/plugin_security.py:1496
↓ 1 callersFunction_canary_client_text_indexes
Argument indexes of a curl or wget command line that are not a file it reads: a ``-d``/``--data`` value without ``@`` (``--data-raw`` always), a `
src/skillevaluator/tier3/eval_core/plugin_security.py:1270
↓ 1 callersFunction_canary_code_loopback
Every network call in code gets a literal loopback URL.
src/skillevaluator/tier3/eval_core/plugin_security.py:1543
↓ 1 callersFunction_canary_code_sinks
Sink kinds for a whole program a code tool runs: a value read on one line can be sent on the next.
src/skillevaluator/tier3/eval_core/plugin_security.py:1928
↓ 1 callersFunction_canary_command_sinks
Sink kinds, outside paths, and read-back candidates for one shell command, one statement part at a time. ``state`` carries the tainted files and
src/skillevaluator/tier3/eval_core/plugin_security.py:1912
↓ 1 callersFunction_canary_env_dump
(name, args, spec)
src/skillevaluator/tier3/eval_core/plugin_security.py:1192
↓ 1 callersFunction_canary_env_lookup
An ``awk`` ``ENVIRON["NAME"]``, ``jq`` ``env.NAME`` or ``$ENV.NAME``, or ``yq`` ``env(NAME)`` read.
src/skillevaluator/tier3/eval_core/plugin_security.py:697
↓ 1 callersFunction_canary_env_reference
``$NAME`` or ``${NAME...}`` of the canary variable (``${#NAME}`` is only its length).
src/skillevaluator/tier3/eval_core/plugin_security.py:674
↓ 1 callersFunction_canary_evidence
Evidence with the canary token and any credential in it redacted, cut to the evidence limit.
src/skillevaluator/tier3/eval_core/plugin_security.py:669
↓ 1 callersFunction_canary_excludes_decoy
An exclusion pattern (``--exclude``, ``-path ... -prune``) covers the decoy directory or file.
src/skillevaluator/tier3/eval_core/plugin_security.py:779
↓ 1 callersFunction_canary_file_has_token
(path, spec)
src/skillevaluator/tier3/eval_core/plugin_security.py:3878
↓ 1 callersFunction_canary_function_definition
``(name, body)`` when ``unit`` defines a shell function (``f() { ...; }``, ``function f { ...; }``).
src/skillevaluator/tier3/eval_core/plugin_security.py:1709
↓ 1 callersFunction_canary_git_records
The git subcommand records something that can carry the canary: a tag it creates (not one it lists, verifies or deletes), a note it adds, a stash
src/skillevaluator/tier3/eval_core/plugin_security.py:1247
↓ 1 callersFunction_canary_git_rest
The arguments after the git subcommand.
src/skillevaluator/tier3/eval_core/plugin_security.py:1233
↓ 1 callersFunction_canary_inline_payload
``(lead, assignments, payload, trailing)`` of ``sh -c``, ``eval``, or a heredoc fed to a shell, else ``None``.
src/skillevaluator/tier3/eval_core/plugin_security.py:913
↓ 1 callersFunction_canary_link
Record ``ln -s target link``: a link to the decoy or a tainted file is tainted, and paths through it resolve to its target. A relative target is r
src/skillevaluator/tier3/eval_core/plugin_security.py:1691
↓ 1 callersFunction_canary_names_root
A shell word resolves to a workspace root, or to a directory that holds one.
src/skillevaluator/tier3/eval_core/plugin_security.py:766
↓ 1 callersFunction_canary_patch_writes
``(path, carried)`` for each file an ``apply_patch`` command's patch writes. The patch is the command's argument or heredoc, or, when it has none
src/skillevaluator/tier3/eval_core/plugin_security.py:1362
↓ 1 callersFunction_canary_piped_script
Literal ``echo``/``printf`` text that ``producer | <shell>`` runs as a script, else ``None``. The shell reads its script from standard input: no
src/skillevaluator/tier3/eval_core/plugin_security.py:965
↓ 1 callersFunction_canary_redirect_words
The redirection operators and their targets among ``args``.
src/skillevaluator/tier3/eval_core/plugin_security.py:956
↓ 1 callersFunction_canary_remote
``rsync`` and ``scp`` reach the network only with a remote operand; ``rsync -a ./ /tmp/x`` is a local copy.
src/skillevaluator/tier3/eval_core/plugin_security.py:1467
↓ 1 callersFunction_canary_substitutions
Bodies of the ``$( ... )`` and backtick command substitutions in ``text`` (outermost, bounded).
src/skillevaluator/tier3/eval_core/plugin_security.py:1161
↓ 1 callersFunction_canary_symlinks
``(target, link)`` pairs an ``ln -s`` command line makes; a link into a directory gets both forms.
src/skillevaluator/tier3/eval_core/plugin_security.py:1324
↓ 1 callersFunction_canary_tool_kind
The sink kind of a tool call by its name; ``bare`` lets a bare unknown name be an MCP call.
src/skillevaluator/tier3/eval_core/plugin_security.py:1426
↓ 1 callersFunction_canary_unit_sinks
Read one statement part; add sink kinds to ``found`` and update the taint state. ``found`` holds ``kinds`` (a list), ``paths`` (outside files tha
src/skillevaluator/tier3/eval_core/plugin_security.py:1722
↓ 1 callersFunction_canary_write_body
(args)
src/skillevaluator/tier3/eval_core/plugin_security.py:1944
↓ 1 callersFunction_canonical_anthropic_authority
(netloc: str, hostname: str)
src/skillevaluator/provider_config.py:743
↓ 1 callersFunction_canonical_anthropic_authority
(netloc, hostname)
src/skillevaluator/tier3/harbor/templates/eval.py:1494
↓ 1 callersFunction_canonical_anthropic_path
(path: str)
src/skillevaluator/provider_config.py:804
↓ 1 callersFunction_canonical_anthropic_path
(path)
src/skillevaluator/tier3/harbor/templates/eval.py:1555
↓ 1 callersFunction_canonical_json
(value: Any, root_template: str | None = None)
src/skillevaluator/tier3/loose_parts.py:1316
↓ 1 callersFunction_canonical_output_binding
(path: Path)
src/skillevaluator/tier3/output_provenance.py:497
↓ 1 callersFunction_canonical_roots
Undo only the loose arm's recorded substitutions, at their original fields.
src/skillevaluator/tier3/loose_parts.py:696
↓ 1 callersFunction_canonical_tier3_embed
Return one safe canonical Tier 3 copy and its browser decoding mode.
src/skillevaluator/reporting/html.py:1650
↓ 1 callersFunction_canonicalize_platform_root_alias
Resolve only a trusted root-owned POSIX alias such as macOS /tmp.
src/skillevaluator/tier3/case_ids.py:80
↓ 1 callersFunction_canonicalize_platform_root_alias
Normalize only an immutable, administrator-owned first-level alias. macOS exposes normal temporary paths below the root-owned ``/var`` link.
src/skillevaluator/tier3/harbor/secure_copy.py:160
↓ 1 callersFunction_capped_tier3_trial_rows
Count the drill-down rows the bounded preview walk leaves out. The walk keeps the first ``_TIER3_HTML_PREVIEW_COLLECTION_ITEMS`` entries of e
src/skillevaluator/reporting/html.py:1710
↓ 1 callersFunction_captured_text
Normalize partial subprocess output without dropping undecodable bytes.
src/skillevaluator/utils/tool_runner.py:121
↓ 1 callersFunction_card_comparison_basis
Explain which cases an evaluator card's lift left out while its arm scores keep them. ``excluded_cases`` is the comparison's own count. ``case_id
src/skillevaluator/evaluation/tier3_report.py:3312
↓ 1 callersFunction_carries_a_nested_invocation
Whether an invocation of the script sits inside an unrecognised command. `flock /tmp/lock python run.py`, `strace -f python run.py` and `task
src/skillevaluator/tier3/eval_core/script_invocation.py:1470
↓ 1 callersFunction_case_attempt_coverage
(case_id: str, attempts: list[dict[str, Any]])
src/skillevaluator/tier3/harbor/collector.py:7046
↓ 1 callersFunction_case_contract
(result: Mapping[str, Any], run_dir: Path)
src/skillevaluator/tier3/interactions.py:71
↓ 1 callersFunction_case_expected
Return the attempts a case should have run, from its pass@k record. The collector counts the case's scored, recovered and missing attempts, w
src/skillevaluator/tier3/token_usage.py:319
↓ 1 callersFunction_case_first_summary
(result: Mapping[str, Any], *, console: Console, skill_path: Path | None)
src/skillevaluator/tier3/result_display.py:1095
↓ 1 callersFunction_case_first_turn_tokens
(trials: Sequence[TrialObservation])
src/skillevaluator/tier3/harbor/stats.py:1308
↓ 1 callersMethod_case_id
(self, root: Path, row: Mapping[str, Any] | None, result: Any)
src/skillevaluator/tier3/plugin_evidence.py:364
↓ 1 callersFunction_case_id_from_trial_dir
Resolve eval case id from persisted Harbor metadata, else the folder name.
src/skillevaluator/tier3/generate_dataset.py:1025
↓ 1 callersFunction_case_id_validation
Say whether every case row names a canonical dataset case, and name the rows that do not. Nothing is recorded without a dataset to check against:
src/skillevaluator/tier3/token_usage.py:617
↓ 1 callersFunction_case_insensitive
(path: Path)
tests/tier3/test_decoys.py:252
↓ 1 callersFunction_case_line_rows
(label: str, states: dict[str, str], case_ids: list[str], prefix: str)
src/skillevaluator/reporting/console_ui.py:326
↓ 1 callersFunction_case_lookup
(dataset: list[dict])
src/skillevaluator/evaluation/insights_judge.py:261
↓ 1 callersFunction_case_pattern_end
Return the index after a ``case`` arm pattern at ``idx``, or ``None``. ``x)``, ``*)``, ``"$1")``, ``(x)``, ``( x)`` and ``x )`` each end a pa
src/skillevaluator/tier3/eval_core/checks.py:2571
↓ 1 callersMethod_case_phrases
(self, k: int)
src/skillevaluator/evaluation/dataset_case_removal.py:425
← previousnext →5,701–5,800 of 24,741, ranked by callers