Helper method to get the output value from an item using the first output field

Get the current lambda penalty value.

Retrieve a session by ID.

Log an LLM interaction with structured data. Args: prompt (str): The prompt sent to the LLM response (str): The response rece

Configure the MetricsManager with output fields. Args: output_fields (List[str]): List of output field names

Process an initial optimization request.

Clean and format LLM response.

Add a new log entry. Args: entry_type (str): Type of log entry (e.g., "ERROR", "INFO") data (Dict[st

Convert config to a serializable dictionary.

Convert feedback to dictionary format.

Call LLM API directly based on the configured provider. Args: prompt (str): The prompt to send to the LLM

Prepare a dataset from input data.

Run the optimization process using the configured backend. Args: initial_flag (bool): Whether this is the initia

Update the latest optimized prompt.

Calculate sizes for training and validation datasets.

Develop prompt template components from raw input. Flexibly extracts components from raw_input regardless of their order.

Evaluate a prompt using the meta-prompt backend by testing it against synthetic data. Args: prompt (str): The pr

Extract input and output fields from task description and sample data. Args: tmp_lm: Language model instance Retu

Extract or set task description from human input. Args: tmp_lm: Language model instance Returns: str:

Extract task type from task description and sample data. Args: tmp_lm: Language model instance Returns:

Extract tools for agentic/programming tasks. Args: tmp_lm: Language model instance Returns: Optional[

Parse field definitions from string or list.

Parse input fields from config.

Prepare the final results dictionary.

Process and simplify human feedback if present in input. Args: tmp_lm: Language model instance Returns:

Save session data to a file.

Set appropriate DSPy module based on task description and sample data. Args: tmp_lm: Language model instance Retu

Configure parameters based on search type. Sets data and optimizer related parameters according to the search strategy (quick

Set trainer based on task type.

Set up model configuration based on provider. Returns: dspy.LM: Configured language model instance Raise

Add a new feedback to the store.

Calculate comprehensive quality scores.

Generate synthetic training data based on sample data in batches.

Get all feedback for a specific prompt.

Get final evaluation metrics for the task type.

Get comprehensive performance summary.

Get summary statistics.

Parse and validate command line arguments. Returns: Dict: Configuration dictionary with all parsed arguments

Convert session to dictionary format.

Apply common preprocessing steps to both datasets.

Apply dataset-specific configurations and preprocessing.

Call Anthropic API directly. Args: prompt (str): The prompt to send Returns: st

Call OpenAI API directly. Args: prompt (str): The prompt to send Returns: str:

Clean and format dictionary values.

Clean parsed JSON data.

Compile the program using the trainer.

Create a prediction object with the expected structure for evaluation. Args: prediction_text (str): Raw predicti

Create sample data from available datasets. Args: train_dataset: HuggingFace Dataset for training test_datase

Generate a high-quality prompt for synthetic data creation with specified batch size.

Create task description from dataset.

Create a test input string from a sample data dictionary. Args: sample (Dict): Sample data containing input fiel

Extract sample data from task description and human input. Args: tmp_lm: Language model instance Returns:

Initialize the DSPy trainer.

Load and preprocess dataset from local files or HuggingFace. Returns: tuple[Dataset, Dataset]: Processed (train_dataset,

Load datasets from HuggingFace.

Load datasets from local CSV files.

Manually parse dictionary-like strings.

Populate missing configuration parameters in a specific order. This method handles the core configuration flow: 1. Sets searc

Populate configuration from HuggingFace dataset.

Prepare sample, training, and validation datasets. This method handles: 1. Creating sample data from available datasets

Prepare training and validation datasets.

Prepare full validation dataset if available.

Prepare sample data for synthetic data generation.

Prepare training and validation dataset splits. Args: train_dataset: HuggingFace Dataset for training test_da

Process dictionary pairs into a clean dictionary.

Special processing for GSM8K dataset to handle math reasoning steps. Args: train_dataset: HuggingFace Dataset for trainin

Special processing for SQuAD dataset.

Run optimization using DSPy backend. Args: initial_flag (bool): Whether this is the initial optimization

Run optimization using meta-prompt backend with direct API calls. Args: initial_flag (bool): Whether this is the

Split dictionary content into key-value pairs.

Validate the generated synthetic data for quality and consistency. Args: data (Dict): The generated data sample

Add optimization result for tracking.

Calculate cost efficiency score.

Calculate clarity score based on prompt characteristics.

Calculate quality score for synthetic data.

Calculate time efficiency score.

Create and store a new optimization session.

Create a DSPy signature for the optimization task. Args: name (str): Name of the signature input_fie

Display optimization results in a fancy, formatted way. Args: result (Dict): The result dictionary from process_input

Download required NLTK data during installation.

Export results to CSV file.

Export metrics to JSON file.

Export results to JSON file.

Export feedback to a JSON file.

Creates a prompt that instructs an LLM to extract both input and output fields from sample data based on the task. This combined approach is

Generate comprehensive feedback for an optimized prompt using synthetic data with explicit arguments. This function: 1. Uses the pro

Return a meta‑prompt that instructs a downstream optimizer to improve the given `initial_prompt` while strictly preserving immutable schema b

Generate analytical feedback for AI system prompt optimization. Args: user_input (str): Original user input/request ai_s