MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 6 callersFunction_result
(generation: dict | None)
tests/reporting/test_dataset_generation_provenance.py:37
↓ 6 callersFunction_reward_judge_models
Return the judge models that scored ``metric`` in one reward row. A RAGAS score is tagged ``"<model> (ragas)"`` so it never pairs with a prom
src/skillevaluator/tier3/harbor/collector.py:5543
↓ 6 callersFunction_reward_multi_metric
Reward with evidence_refs on multiple metrics for lookup testing.
tests/tier3/test_suggestion_grounding.py:338
↓ 6 callersFunction_rm_shell_tokens
Keep rm operands separate from comments and backtick command payloads.
src/skillevaluator/tier3/eval_core/checks.py:5438
↓ 6 callersFunction_rm_wrapped_command_index
Find a literal command behind the common transparent shell wrappers.
src/skillevaluator/tier3/eval_core/checks.py:5560
↓ 6 callersFunction_root_ref
The root placeholder *token* starts with (followed by a separator or nothing), or ``None``.
src/skillevaluator/plugin_component_risk.py:1721
↓ 6 callersFunction_run
(root)
tests/reporting/test_applicability_saved_runs.py:45
↓ 6 callersFunction_run_engine
Run ``run_harbor_eval`` over five staged cases with the Harbor edges mocked.
tests/test_tier3_max_cases.py:40
↓ 6 callersFunction_runtime_context_is_bound_to_publication_target
Check that every staged runtime source is covered by the skill digest. ``publication_target`` deliberately identifies only the evaluated target
src/skillevaluator/tier3/harbor/runner.py:242
↓ 6 callersFunction_safe_scalar_text
(value: object)
src/skillevaluator/reporting/benchmark.py:1409
↓ 6 callersMethod_scan_file_for_pii
Scan a single file for PII patterns, yielding findings with full context. ``protected_usernames`` is the set of author/submitter identities u
src/skillevaluator/validators/security.py:6457
↓ 6 callersFunction_score
(module, command: str)
tests/tier3/test_license_notice_network_bypasses.py:24
↓ 6 callersFunction_score_policy_for_rewards
Recover recorded reward policy; flag a policy shared by equal-score formulas as ambiguous.
src/skillevaluator/tier3/harbor/collector.py:5726
↓ 6 callersFunction_section
(text: str, heading: str)
tests/reporting/test_tier3_token_usage_reports.py:188
↓ 6 callersFunction_section
()
tests/reporting/test_tier3_completed_agent_docs.py:42
↓ 6 callersFunction_section_skill
A skill with one 200-character section per letter, in SKILL.md.
tests/deduplication/intra_skill/test_intra_skill_validator.py:1417
↓ 6 callersFunction_security_evidence_html
(html: str)
tests/reporting/test_tier3_failure_reasons.py:126
↓ 6 callersFunction_session
(*, tool="skill_view", arguments=None, observation=None)
tests/test_hermes_agent.py:70
↓ 6 callersFunction_set_universal_analyzer_work
Keep synthetic complete reports aligned with producer work accounting.
tests/validators/test_security.py:252
↓ 6 callersFunction_shell_c_payload
Extract inline command string from -c shell invocation. The option and its payload are read from the interpreter's own operands, with every r
src/skillevaluator/tier3/eval_core/script_invocation.py:333
↓ 6 callersFunction_skill_manifest
(path: Path)
src/skillevaluator/tier3/harbor/adapter.py:467
↓ 6 callersFunction_skillspector_scoring_source_scope
Return the producer's source identity for score reconciliation.
src/skillevaluator/validators/security.py:1119
↓ 6 callersFunction_source
(tmp_path: Path, manifest_rel: str, skills: Any)
tests/tier3/test_openclaw_prepare.py:218
↓ 6 callersFunction_sse_events
(body: str)
tests/test_nvidia_build_bridge.py:295
↓ 6 callersFunction_stamp_publication_evidence
Stamp the producer contract expected from built-in tier wrappers.
tests/reporting/test_advisory_skip_verdict.py:97
↓ 6 callersMethod_start_bridge
(self, environment: BaseEnvironment)
src/skillevaluator/tier3/harbor/local_agents.py:791
↓ 6 callersFunction_status
(code)
tests/test_retry_exception_chain.py:17
↓ 6 callersFunction_status
(root: Path)
tests/harness/test_plugin_compat.py:37
↓ 6 callersFunction_step
(call_id: str, fn: str, args: dict[str, Any], content: str, **extra: Any)
tests/tier3/test_f90_mcp_arguments.py:23
↓ 6 callersFunction_strip_attempt_suffix
Remove SkillEvaluator per-attempt suffixes from a task/case identifier.
src/skillevaluator/tier3/harbor/collector.py:4566
↓ 6 callersFunction_table_rows
Return ``{first cell: [remaining cells]}`` for the table whose header starts with *heading_cell*.
tests/test_metrics_reference_docs.py:104
↓ 6 callersFunction_tc
(fn: str, args: dict[str, Any] | None = None, call_id: str = "c1")
tests/tier3/test_plugin_signals.py:47
↓ 6 callersFunction_terminal_text
Drop Rich's own SGR styling and fail if any other control character is left.
tests/reporting/test_rich_markup_escaping.py:84
↓ 6 callersFunction_token_limit_kwargs
(config: ProviderConfig, max_tokens: int | None)
src/skillevaluator/inference/client.py:124
↓ 6 callersFunction_traj
(agent_message: str)
tests/tier3/test_harbor_template_secret_patterns.py:61
↓ 6 callersFunction_traj
(*steps: dict[str, Any])
tests/tier3/test_f90_mcp_arguments.py:35
↓ 6 callersFunction_trajectory_dict_steps
(trajectory: dict[str, Any])
src/skillevaluator/tier3/harbor/collector.py:1778
↓ 6 callersFunction_tree
(root: Path)
tests/tier3/test_private_loose_source.py:97
↓ 6 callersFunction_tree
(root: Path, files: dict)
tests/tier3/test_plugin_prepare.py:25
↓ 6 callersFunction_trial
(tmp_path, *, agent_result=None, steps=None, failed=False)
tests/tier3/test_private_usage_recovery_review2.py:28
↓ 6 callersFunction_truncate_for_behavior
(text: str, limit: int)
src/skillevaluator/tier3/eval_core/atif_helpers.py:329
↓ 6 callersFunction_unit_interval_score
Return a finite score in the documented 0.0 to 1.0 range.
src/skillevaluator/evaluation/tier3_report.py:138
↓ 6 callersFunction_unsupported_native_codex_result
(reason: str)
src/skillevaluator/tier3/eval_core/checks.py:6992
↓ 6 callersFunction_update_raw_html_stack
Track Markdown nested inside raw HTML containers across block tokens.
scripts/ci/check_public_benchmarks.py:1492
↓ 6 callersFunction_validate_entry
(metadata: os.stat_result, path: Path, root_device: int | None)
src/skillevaluator/publication_identity.py:106
↓ 6 callersFunction_value_now
The value an assignment stores: each variable it reads, expanded now. An assignment copies the value it reads; it is not a live alias, so ``f
src/skillevaluator/tier3/eval_core/script_invocation.py:995
↓ 6 callersFunction_verifier
Point the Harbor verifier at tmp_path with stubbed trajectory, tool calls, security and judges.
tests/tier3/test_skill_metric_applicability.py:30
↓ 6 callersFunction_verifier_env_vars
Return evaluator-owned variables explicitly staged for the verifier.
src/skillevaluator/tier3/harbor/adapter.py:428
↓ 6 callersFunction_write
(path, value)
tests/test_tier3_history.py:22
↓ 6 callersFunction_write
(root: Path, files: dict[str, str | bytes | dict | list])
tests/validators/test_plugin_static_risk_review_fixes.py:31
↓ 6 callersMethod_write
(self, body: bytes)
tests/test_harbor_runtime_preflight.py:1917
↓ 6 callersFunction_write_current_harbor_attempt
( job_dir: Path, *, trial_name: str, root_completed: int, root_total: int = 1, result_
tests/test_harbor_runner_status.py:1653
↓ 6 callersFunction_write_current_run
(root: Path, name: str = "20260709_120000_1_aaaaaaaaaaaa")
tests/test_results_location_legacy_compatibility.py:112
↓ 6 callersFunction_write_dataset_skill
(tmp_path: Path, entry: dict, *, dirname: str = "release-review", name: str | None = None)
tests/tier3/test_dataset_baseline_fairness.py:1938
↓ 6 callersFunction_write_explicit_empty_entry
(target: Path)
tests/test_harbor_runtime_skill_isolation.py:3941
↓ 6 callersFunction_write_generated_skill
(tmp_path: Path, entry_ids: list[str])
tests/test_harbor_adapter_safety.py:84
↓ 6 callersFunction_write_test_sh
(task_dir: Path, *, grading_mode: str, custom_grader: bool)
src/skillevaluator/tier3/harbor/adapter.py:2385
↓ 6 callersFunctionadapter
()
tests/ci/test_benchmark_publish.py:15
↓ 6 callersFunctionadditional_warnings
Return warnings not already displayed through a structured finding. Medium and low findings also populate the legacy warning list. Some scann
src/skillevaluator/reporting/base.py:2571
↓ 6 callersFunctionagent_declarative_checks
Omit absent arms and omit the entire section for non-declarative runs.
src/skillevaluator/tier3/declarative_coverage.py:66
↓ 6 callersFunctionaggregate_declarative_checks
Keep every check and each reward's denominator; never pool behavior scores.
src/skillevaluator/tier3/declarative_coverage.py:16
↓ 6 callersFunctionanalyze_plugin
(root: Path, policy: AnalysisOptions | None = None)
tests/validators/plugin_analysis_support.py:56
↓ 6 callersMethodapi
(self, method, suffix="", payload=None, *, missing_ok=False)
src/skillevaluator/_ci_onboarding_all_tiers.py:263
↓ 6 callersMethodarm_for
(self, agent: str)
src/skillevaluator/tier3/harbor/loose_parts_arm.py:110
↓ 6 callersMethodat_end
(self)
src/skillevaluator/tier3/eval_core/codex_tool_call_normalizer.py:903
↓ 6 callersFunctionbaseline_description
Render a compact provenance line shared by reports.
src/skillevaluator/tier3/baseline_store.py:386
↓ 6 callersFunctionbuild_plugin_signals_context
Build a bounded :class:`PluginSignalsContext` from dataset case entries. ``plugin_name`` defaults to the name in the generated wrapper package
src/skillevaluator/tier3/eval_core/plugin_signals.py:1411
↓ 6 callersFunctionbundle_entry
(path, *, name="bundle", fingerprint="c" * 64, members=("same-skill",))
tests/deduplication/plugin/test_private_dedup.py:128
↓ 6 callersFunctioncall
(call_id, name, args=None, *, refs=None)
tests/tier3/test_plugin_evidence_review.py:15
↓ 6 callersFunctionchat_completion_to_anthropic_events
Yield terminal Anthropic Messages stream events for one Chat Completion.
src/skillevaluator/tier3/harbor/nvidia_build_bridge.py:1321
↓ 6 callersFunctioncomparison_identity
Keep unknown configuration relevant by default, separating skill identity.
src/skillevaluator/tier3/baseline_store.py:137
↓ 6 callersMethodcomponent_modes
(self)
src/skillevaluator/tier3/plugin_native.py:1058
↓ 6 callersFunctioncopy_file_secure
Copy one authored regular file with atomic overwrite and rollback. The rollback guarantee assumes the source, destination, private stage, and
src/skillevaluator/tier3/harbor/secure_copy.py:2468
↓ 6 callersMethodcreate
(self, **kwargs)
tests/test_llm_judge.py:925
↓ 6 callersMethodcreateResolver
(t,e,i=[""],s)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 6 callersFunctioncreate_skill_with_declared_license
Create a skill whose only license signal is its SKILL.md license field.
tests/validators/test_license.py:271
↓ 6 callersFunctioncvss_to_severity
Convert CVSSv3 score to severity level.
src/skillevaluator/utils/tool_runner.py:52
↓ 6 callersFunctiondefault_runtime_root
Return the validated runtime root, honoring its environment override.
src/skillevaluator/tier3/harbor/local_runtime.py:95
↓ 6 callersFunctiondetail_row
A plain label/value row, with an optional muted note.
src/skillevaluator/reporting/console_ui.py:270
↓ 6 callersMethoddigest
Return a stable digest for the effective, source-independent policy.
src/skillevaluator/validators/policy.py:85
↓ 6 callersMethoddraw
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 6 callersMethodembed_chunked
Split text into chunks, embed each, and average-pool. Used by --full-body mode for documents that may exceed the model's context wind
src/skillevaluator/embedding/client.py:264
↓ 6 callersFunctionencode_environment_kwarg
Encode one validated kwarg so Harbor's parser preserves its JSON type.
src/skillevaluator/tier3/evals_config.py:756
↓ 6 callersFunctionextract_from_rule
Extract title + description from a .mdc rules file.
src/skillevaluator/embedding/extractor.py:191
↓ 6 callersFunctionf
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:7
↓ 6 callersFunctionfn
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 6 callersMethodgetDistanceFromCenterForValue
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 6 callersMethodgetMinMax
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 6 callersMethodgetPixelForDecimal
(t)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 6 callersMethodgetStyle
(t,e)
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 6 callersFunctiongit
(repo: Path, *args: str, optional: bool = False, network: bool = False)
src/skillevaluator/_ci_onboarding_git.py:52
↓ 6 callersFunctiongoal_judge
(verifier)
tests/tier3/test_judge_policies.py:109
↓ 6 callersFunctionharness_panel
One panel: a ✓/!/✗ line per skill, then its worst finding per harness (every finding with ``verbose``).
src/skillevaluator/harness/report.py:194
↓ 6 callersFunctionignore
(directory: str, names: list[str])
src/skillevaluator/tier3/harbor/runner.py:4462
↓ 6 callersMethodindependent_errors
Return errors no structured finding repeats, such as execution diagnostics.
src/skillevaluator/models/result.py:652
↓ 6 callersFunctioninstall_local_runtime
Install ``agent``'s pinned CLI under the runtime root; return an error, or ``None`` on success.
src/skillevaluator/tier3/harbor/local_runtime.py:530
↓ 6 callersMethodissue
(self, source: str, detail: object)
src/skillevaluator/reporting/plugin_runtime.py:82
↓ 6 callersMethoditems
(self)
src/skillevaluator/utils/structured_data.py:287
↓ 6 callersFunctioniter_candidate_results_roots
Return read candidates in precedence order, with legacy fallback. Read commands should honor the same primary resolution as write commands, b
src/skillevaluator/tier3/results_location.py:99
↓ 6 callersFunctionkill_quietly
(pids)
tests/ci/test_run_tier1.py:1537
← previousnext →1,501–1,600 of 24,741, ranked by callers