Result of constraint validation.

A single evaluation example.

Train/val/holdout split of evaluation examples.

Configuration for a self-evolution optimization run.

Validates evolved artifacts against hard constraints.

Multi-dimensional fitness score.

Use LLM-as-judge to determine which messages are relevant to a skill. Two-stage pipeline: 1. Cheap heuristic pre-filter (_is_relevant_to_sk

A DSPy module that wraps a skill file for optimization. The skill text (body) is the parameter that GEPA optimizes. On each forward pass, the

Generate evaluation datasets using a strong LLM. Reads the target artifact (skill file, tool description, etc.) and generates realistic (task

Import user prompts from Claude Code history.jsonl. Claude Code stores a flat JSONL of user messages at ~/.claude/history.jsonl. Each line ha

Import conversations from GitHub Copilot session events. Copilot stores sessions at ~/.copilot/session-state/<session-id>/. Each session has

Generate realistic evaluation test cases for an agent skill or tool. Given the full text of a skill/tool description, generate diverse test c

Load hand-curated evaluation datasets from JSONL files.

Import conversations from Hermes Agent session files. Hermes stores session transcripts as JSON files in ~/.hermes/sessions/. Each file conta

Evaluate an agent's response against an expected behavior rubric. Score the response on three dimensions (0.0 to 1.0 each): 1. correc

LLM-as-judge scorer with rubric-based evaluation. Scores agent outputs on multiple dimensions and provides textual feedback that GEPA can use

Score whether a user message is relevant to a specific agent skill. Return a JSON object with: - relevant: boolean (true if the messa

Complete a task following the provided skill instructions. You are an AI agent following specific skill instructions to complete a task.

Test the main orchestration function.

Test the Click CLI entry point using CliRunner.

Verify the full pipeline: fake files -> import -> filter -> save -> reload. This is the most important test. It proves that output from build

Verify MIN_DATASET_SIZE warning in build_dataset_from_external.

Test RelevanceFilter with mocked LLM calls.

Verify cheap pre-filter catches obvious matches and rejects non-matches.

Verify _parse_scoring_json handles various LLM output formats.

Verify that known secret formats are caught and normal text is not.

Verify that short skill names match via exact full-name check.

Verify _validate_eval_example normalizes, rejects, and caps fields.

Verify validation is wired correctly into RelevanceFilter.