MCPcopy Create free account

hub / github.com/NVIDIA/SkillEvaluator / functions

Functions24,741 in github.com/NVIDIA/SkillEvaluator

↓ 1 callersFunction_matching_entries
The names in ``directory`` that ``wanted`` accepts, in name order (the listing is never kept whole).
src/skillevaluator/plugin_manifest.py:418
↓ 1 callersFunction_materialize_embedded_trajectory
( trajectory: dict[str, Any], *, agent_dir: Path, state: _TrajectoryReferenceState, depth:
src/skillevaluator/tier3/harbor/collector.py:2528
↓ 1 callersFunction_materialized_trajectory_source_ids
( trajectory: dict[str, Any], *, state: _TrajectoryReferenceState, )
src/skillevaluator/tier3/harbor/collector.py:2511
↓ 1 callersMethod_maxDigits
()
src/skillevaluator/reporting/assets/chart.umd.min.js:13
↓ 1 callersFunction_max_detail_pass_summary
()
tests/test_harbor_runner_artifacts.py:118
↓ 1 callersFunction_max_detail_result
(run_dir: Path)
tests/test_harbor_runner_artifacts.py:165
↓ 1 callersFunction_may_be_header_file_item
Whether an HTTPie argument may reach it as a ``name:@file`` header item. The shell removes quotes and backslashes, so ``X-Data:"@.env"``, ``"
src/skillevaluator/tier3/eval_core/checks.py:2200
↓ 1 callersFunction_mcnemar_exact_p_value
Return the two-sided exact McNemar p-value, or None on float underflow.
src/skillevaluator/tier3/harbor/collector.py:4958
↓ 1 callersMethod_mcp
Write the plain MCP config: the plugin arm's ``.mcp.json`` entries with the root variables replaced.
src/skillevaluator/tier3/loose_parts.py:911
↓ 1 callersFunction_mcp_call_targets_root
Whether an MCP call searches or lists the whole filesystem. Only filesystem tools count: a tool of a server whose name contains ``filesystem`
src/skillevaluator/tier3/eval_core/checks.py:8303
↓ 1 callersFunction_mcp_evidence
( server: str, plugin: str, statuses: Mapping[str, str], tools: set[str], calls: Mapping[s
src/skillevaluator/tier3/loose_parts.py:1630
↓ 1 callersFunction_mcp_ident
The MCP identity of a call named ``fn``, labeled ``mcp__<server>__<tool>`` (see :meth:`_McpNames.identity`).
src/skillevaluator/tier3/eval_core/plugin_signals.py:2811
↓ 1 callersFunction_mcp_statuses
Server name -> the best status any init event reported (``connected`` wins).
src/skillevaluator/tier3/loose_parts.py:1526
↓ 1 callersFunction_md
(value)
ci/tier_manifest.py:302
↓ 1 callersFunction_mean
(values: Sequence[float])
src/skillevaluator/tier3/eval_core/plugin_signals.py:4831
↓ 1 callersFunction_member
(value: str)
src/skillevaluator/deduplication/plugin/profile.py:72
↓ 1 callersFunction_member_manifest_match
(path: str, members: Sequence[str])
src/skillevaluator/tier3/eval_core/plugin_signals.py:2629
↓ 1 callersFunction_member_text
(reader: PluginRootReader, manifest: Mapping[str, Any], member: PluginMember)
src/skillevaluator/tier3/plugin_prepare.py:549
↓ 1 callersFunction_members
(values)
src/skillevaluator/deduplication/plugin/llm_analyzer.py:35
↓ 1 callersFunction_mentions_skill_artifact
(value: Any)
src/skillevaluator/tier3/eval_core/checks.py:1897
↓ 1 callersFunction_merge_across_cases
Collapse one cause repeated across cases (same error type, or same kind and scope) into one item.
src/skillevaluator/tier3/case_summary.py:1328
↓ 1 callersFunction_merge_alias_dag
(levels: int)
tests/utils/test_structured_data.py:51
↓ 1 callersFunction_merge_constituent_judge_models
Keep the scoring steps' judges without replacing Harbor's aggregate scores.
src/skillevaluator/tier3/harbor/collector.py:5508
↓ 1 callersFunction_merge_split_pieces
Report pieces of one split source chunk once, with their merged range.
src/skillevaluator/deduplication/intra_skill/semantic_clustering.py:68
↓ 1 callersFunction_merge_trajectory_final_metrics
( trajectories: list[dict[str, Any]], *, continuation_flags: list[bool], total_steps: int, )
src/skillevaluator/tier3/harbor/collector.py:3009
↓ 1 callersFunction_merge_trial_evaluation_failures
Preserve judge-failure diagnostics when Harbor supplies an aggregate reward.
src/skillevaluator/tier3/harbor/collector.py:3169
↓ 1 callersFunction_merged_default_mcp_note
LOW: a format's root MCP file is checked next to a declared ``mcpServers`` (Cursor).
src/skillevaluator/plugin_mcp.py:441
↓ 1 callersFunction_message_rejects_schema_option
Match a rejection of the schema option itself, not unrelated error text.
src/skillevaluator/inference/client.py:239
↓ 1 callersFunction_message_rejects_schema_option
Match a rejection of the schema option itself, not unrelated error text.
src/skillevaluator/tier3/harbor/templates/eval.py:1902
↓ 1 callersFunction_metadata_duration_seconds
Read ``duration_seconds`` from the structured tool metadata Harbor keeps for Codex.
src/skillevaluator/tier3/harbor/trajectory_timing.py:223
↓ 1 callersFunction_metadata_field_value
(text: str, field: str)
scripts/ci/check_public_benchmarks.py:1164
↓ 1 callersFunction_metric_label
(metric: str)
src/skillevaluator/tier3/harbor/report.py:271
↓ 1 callersFunction_metric_labels
(ae: dict[str, Any] | None)
src/skillevaluator/reporting/benchmark.py:2120
↓ 1 callersFunction_metric_labels
(metric_ids: list[str])
src/skillevaluator/evaluation/tier3_report.py:4853
↓ 1 callersFunction_metric_name_is_publishable
(name: object)
src/skillevaluator/tier3/harbor/templates/custom_grader_runner.py:177
↓ 1 callersFunction_metric_name_shape_is_valid
(name: object)
src/skillevaluator/tier3/harbor/templates/custom_grader_runner.py:205
↓ 1 callersFunction_metric_signals
(ae: dict[str, Any] | None)
src/skillevaluator/reporting/benchmark.py:2096
↓ 1 callersFunction_metrics_for_rewards
(rewards: list[dict[str, Any]])
src/skillevaluator/tier3/harbor/report_data.py:527
↓ 1 callersFunction_metrics_for_rewards
(rewards: list[dict])
src/skillevaluator/tier3/harbor/templates/metric.py:39
↓ 1 callersFunction_minimum_attainable_mcnemar_p_value
Return the smallest two-sided exact p-value attainable for this pair count.
src/skillevaluator/tier3/harbor/collector.py:4970
↓ 1 callersFunction_mint_embedded_trajectory_id
( reference_key: str, trajectory: dict[str, Any], *, reference_ordinal: int, )
src/skillevaluator/tier3/harbor/collector.py:2494
↓ 1 callersFunction_missing_attempts
(agents: Mapping[str, Mapping[str, Any]])
src/skillevaluator/tier3/next_steps.py:85
↓ 1 callersFunction_missing_gke_kwargs
Return required GKE kwargs that are missing or empty.
src/skillevaluator/tier3/harbor/runner.py:1623
↓ 1 callersFunction_missing_replay_path
(call: dict, fixtures: dict[str, str], skill_name: str, skill_path: Path | None)
src/skillevaluator/tier3/trace_import.py:197
↓ 1 callersFunction_missing_tool
(command: str)
scripts/script_invocation_differential.py:1264
↓ 1 callersFunction_mixed_logical_overall_lift
Compare complete mixed rewards only when both arms scored the same case contracts.
src/skillevaluator/evaluation/tier3_report.py:2326
↓ 1 callersFunction_modal_custom_config_status
()
src/skillevaluator/tier3/harbor/runner.py:2541
↓ 1 callersFunction_modifier_start
Start of the verb modifier ("first", "having to", "being forced to") that ends at ``end``, if any.
src/skillevaluator/evaluation/dataset_case_removal.py:77
↓ 1 callersFunction_module_available
(module: str)
src/skillevaluator/doctor.py:116
↓ 1 callersFunction_module_path_names_exact_version
Whether a remote module URL names an exact version in its path (``/x/mod@v1.2.3/mod.ts``). Only the path counts, the part the server reads: the q
src/skillevaluator/validators/mcp_static.py:2441
↓ 1 callersFunction_monitor_bedrock_metadata_unavailability
Retain transient metadata failures that Botocore may collapse to no credentials.
src/skillevaluator/tier3/harbor/runtime_preflight.py:289
↓ 1 callersFunction_multi_step_timeout_trial
Write a native multi-step trial as Harbor 0.24 records a step's agent timeout. ``MultiStepTrial._run_step_agent`` stores the exception on the ste
tests/tier3/test_baseline_timeouts.py:557
↓ 1 callersMethod_name_starts
Where a case id may start after a noun: directly, after ":", or after link words like "for the".
src/skillevaluator/evaluation/dataset_case_removal.py:350
↓ 1 callersFunction_named_skills
Skills under a folder and their names; one that cannot be read is left out.
src/skillevaluator/harness/overlay.py:695
↓ 1 callersFunction_names_link
(parent_descriptor: int, name: str)
src/skillevaluator/tier3/harbor/secure_copy.py:845
↓ 1 callersFunction_names_recorded_skill
Whether ``skill_path`` names the skill the run recorded. Runs record the NFC directory entry name. The name is accepted before or after symli
src/skillevaluator/evaluation/tier3_feedback_chat.py:129
↓ 1 callersMethod_names_root
Whether ``text`` names a root placeholder, also in a parameter expansion (``${CLAUDE_PLUGIN_ROOT:-}``).
src/skillevaluator/plugin_component_risk.py:3349
↓ 1 callersFunction_names_script_anywhere
Whether the expected script's file name appears anywhere in the command text. This is the reference test the checker applied before invocation ev
src/skillevaluator/tier3/eval_core/script_invocation.py:1532
↓ 1 callersFunction_names_skill_script
Whether text names one of the skill's script files, run or not ("checks manifests with review.py"). The text and names are also checked as `tier3
src/skillevaluator/tier3/generate_dataset.py:542
↓ 1 callersMethod_namespace_components
The documented client layouts in a namespace folder, inventoried as their own (unsupported) types.
src/skillevaluator/plugin_discovery.py:1146
↓ 1 callersFunction_native_dedicated_verifier_scopes
Return verify passes whose dedicated image never receives uploaded tests.
src/skillevaluator/tier3/harbor/adapter.py:5374
↓ 1 callersFunction_native_path_has_content
Return whether an authored native-task path can project any payload.
src/skillevaluator/tier3/harbor/adapter.py:5286
↓ 1 callersFunction_native_plugin_agent_import_paths
The private native plugin-loading wrappers, which build on the agents allowed on every backend.
src/skillevaluator/tier3/harbor/runner.py:5103
↓ 1 callersFunction_native_projection_entry_ids
Read native task IDs without following unsafe authored nodes. The secure projection copy validates the complete selected ``harbor/`` tree as
src/skillevaluator/tier3/harbor/adapter.py:6428
↓ 1 callersFunction_native_source_path_is_ignored
Apply the native copy ignore callback to every lexical path component.
src/skillevaluator/tier3/harbor/adapter.py:4725
↓ 1 callersFunction_native_standard_grader_environment_controls
Return task-owned names that can redirect or poison the standard verifier.
src/skillevaluator/tier3/harbor/adapter.py:5427
↓ 1 callersFunction_native_status
(results: list[ValidationResult], tier: int)
ci/benchmark_reports.py:135
↓ 1 callersFunction_native_task_dirs
Return native Harbor task directories from a copied dataset.
src/skillevaluator/tier3/harbor/adapter.py:4710
↓ 1 callersFunction_native_task_workdir
Read and validate the workdir Harbor will use for a native task.
src/skillevaluator/tier3/harbor/adapter.py:5536
↓ 1 callersFunction_native_test_sh
(language: str)
src/skillevaluator/tier3/commands.py:326
↓ 1 callersFunction_native_timing_ms
(steps: list[dict[str, Any]])
src/skillevaluator/tier3/harbor/trajectory_timing.py:166
↓ 1 callersFunction_native_usage
( trial_dir: Path, run_root: Path, warnings: list[str], *, require_steps: bool = False )
src/skillevaluator/reporting/dashboard_data.py:589
↓ 1 callersFunction_near_miss_text
(copies: int)
tests/tier3/test_github_token_redaction.py:70
↓ 1 callersFunction_near_ref_prefix
A known ref type one typo away from ``word`` (``Skil`` -> ``skill``), so the ref could never match.
src/skillevaluator/tier3/eval_core/plugin_signals.py:706
↓ 1 callersFunction_negated_advice
(negators: tuple[tuple[str, bool], ...], case_subject: str)
tests/test_tier3_case_summary.py:415
↓ 1 callersFunction_negated_advice
(negators: tuple[tuple[str, bool], ...], case_subject: str)
tests/evaluation/test_judges.py:913
↓ 1 callersFunction_negative_control_grounding
Return per-case routing type and grounded negative-control failure state.
src/skillevaluator/evaluation/insights_judge.py:426
↓ 1 callersFunction_nemo_gym_doctor_rows
Readiness rows for ``--backend nemo-gym``: the Gym CLI, its sandbox, the policy endpoint and options.
src/skillevaluator/tier3/commands.py:1171
↓ 1 callersFunction_network_client_outside_license_notice
Whether *text* names a network client anywhere but in a license or copyright notice. Only a ``http://`` or ``https://`` URL on a notice's own com
src/skillevaluator/tier3/eval_core/checks.py:2424
↓ 1 callersFunction_network_client_uploads
Check one curl, wget or HTTPie invocation's arguments for upload indicators.
src/skillevaluator/tier3/eval_core/checks.py:2223
↓ 1 callersFunction_network_executable
Name a command word's executable, resolving ``$CURL`` through earlier assignments. A value with several words, as in ``SSH="ssh -i key host"; $SS
src/skillevaluator/tier3/eval_core/checks.py:2156
↓ 1 callersFunction_network_lookup_readings
Return the command as written and each distinct way its lookups resolve. Checking every reading means resolution can only add a finding: a fa
src/skillevaluator/tier3/eval_core/checks.py:2137
↓ 1 callersFunction_network_tokens_exfiltrate
Inspect a tokenized shell command for network client exfiltration indicators.
src/skillevaluator/tier3/eval_core/checks.py:2798
↓ 1 callersFunction_new_work_segments
Drop each Harbor step's copied prefix, or ``None`` if copied steps are not a prefix.
src/skillevaluator/tier3/harbor/trajectory_timing.py:359
↓ 1 callersFunction_newest_run_dir
Return the newest *completed* timestamped run directory under ``root``. This is the cross-platform fallback for when the ``latest`` symlink was
src/skillevaluator/tier3/results_location.py:1045
↓ 1 callersFunction_no_ground_truth_reward
(trial_id: str, *, security: float, findings: list[dict], efficiency: float = 0.9)
tests/test_public_benchmark_gate.py:1204
↓ 1 callersFunction_node_fingerprint
(metadata: os.stat_result)
src/skillevaluator/tier3/output_provenance.py:39
↓ 1 callersFunction_node_package_root
Return a concrete npm package root when ``path`` is below node_modules.
src/skillevaluator/tier3/harbor/local_runtime.py:240
↓ 1 callersFunction_node_source_span
(node: ast.AST, lines: list[str])
scripts/check_oss_boundary.py:347
↓ 1 callersFunction_non_ascii_tier_identifier_lines
Return rows whose Tier Status header or tier label is not ASCII. Generated cards use fixed ASCII identifiers. Restricting only those structur
scripts/ci/check_public_benchmarks.py:1689
↓ 1 callersFunction_nonnegative_counter
(value: Any)
src/skillevaluator/tier3/harbor/report_data.py:602
↓ 1 callersFunction_norm_namespace
(value: str)
src/skillevaluator/tier3/eval_core/plugin_signals.py:2722
↓ 1 callersFunction_normal_cdf
(value: float)
src/skillevaluator/tier3/harbor/stats.py:498
↓ 1 callersFunction_normalise_reasoning_fields
Ensure both ``reasoning_bullets`` and ``explanation`` are populated. The LLM may return either field (or, for cached legacy responses, only `
src/skillevaluator/evaluation/dimension_judge.py:199
↓ 1 callersFunction_normalize
Make compiled patterns / tuples / sequences comparable across modules.
tests/tier3/test_harbor_template_secret_patterns.py:201
↓ 1 callersFunction_normalize_anthropic_base_url
(value, variable)
src/skillevaluator/tier3/harbor/templates/eval.py:1597
↓ 1 callersFunction_normalize_dialect
Map Codex and Agent Plugins MCP entries onto the fields the static policy checks. Codex (``McpServerConfig``): ``type`` is optional and ``streama
src/skillevaluator/plugin_mcp.py:596
↓ 1 callersFunction_normalize_entry
Map agentskills.io field names onto the existing runtime schema.
src/skillevaluator/tier3/dataset_utils.py:147
↓ 1 callersFunction_normalize_harbor_evidence_urls
(raw: object)
src/skillevaluator/evaluation/tier3_report.py:3902
↓ 1 callersFunction_normalize_harbor_upload_summary
(raw: object)
src/skillevaluator/evaluation/tier3_report.py:3856
← previousnext →6,901–7,000 of 24,741, ranked by callers