Get all feedback in dictionary format.

Get evaluation metrics for the task type.

Returns appropriate evaluation metric function based on task type. Args: task_type (str): Type of the task

List all active sessions.

Load sample prompts for demonstration.

Load a specific session from a file path.

Main entry point for the CLI application.

Main CLI entry point.

Main function demonstrating basic Promptomatix usage.

Main function for batch processing demonstration.

Main function for custom metrics demonstration.

Optimize prompt based on feedback for a given session. Args: session_id (str): Session identifier Returns:

Process prompts sequentially.

Run optimization and calculate custom metrics.

Set the lambda penalty value.

Set up dedicated logger for LLM interactions in configuration. Creates a file handler that logs all LLM prompts and responses to a dedicated

Set up dedicated logger for optimization steps and results.

Context manager to suppress stderr output.

Validate synthetic data against the task description using LLM. Args: task (str): Task description or requirements data

Initialize a new session logger. Args: session_id (str): Unique identifier for the session

Initialize configuration parameters for DSPy task. Args: **kwargs: Configuration parameters as described in class attribu

Initialize the optimizer with configuration. Args: config (Config): Configuration object containing optimization

Initialize a new optimization session. Args: session_id (str): Unique identifier for the session ini

Compute metrics for Classification tasks. Returns accuracy (1 for correct, 0 for incorrect). Returns: float: Scor

Compute metrics for Classification tasks. Returns accuracy (1 for correct, 0 for incorrect). Returns: float: Scor

Compute metrics for code completion tasks. Evaluates completion correctness and consistency with context. Returns:

Final evaluation metrics for code completion tasks.

Compute metrics for code debugging tasks. Evaluates bug identification and fix correctness. Returns: float: Score

Final evaluation metrics for code debugging tasks.

Compute metrics for code explanation tasks. Evaluates explanation clarity, completeness, and accuracy. Returns: f

Final evaluation metrics for code explanation tasks.

Compute metrics for code generation tasks. Evaluates code correctness, style, and efficiency. Returns: float: Sco

Final evaluation metrics for code generation tasks.

Compute metrics for conversation tasks. Evaluates response relevance, coherence, and context preservation. Returns:

Final evaluation metrics for conversation tasks.

Compute metrics for data analysis tasks. Evaluates analysis accuracy and insight quality. Returns: float: Score b

Final evaluation metrics for data analysis tasks.

Compute metrics for decision making tasks. Evaluates decision quality and reasoning. Returns: float: Score betwee

Final evaluation metrics for decision making tasks.

Default metrics when task type is unknown. Returns exact match score. Returns: float: Score between 0 and 1

Default metrics when task type is unknown. Returns exact match score. Returns: float: Score between 0 and 1

Final evaluation metrics.

Compute metrics for Text Generation tasks. Returns average of BLEU and ROUGE scores. Returns: float: Score betwee

Compute metrics for Text Generation tasks. Returns average of BLEU and ROUGE scores. Returns: float: Score betwee

Improvise the human input.

Compute metrics for information extraction tasks. Uses F1 score for extracted entities/relations and structure accuracy. Retu

Final evaluation metrics for information extraction tasks.

Compute metrics for multi-label classification tasks. Uses label-based F1 score and Hamming distance. Returns: fl

Final evaluation metrics for multi-label classification tasks.

Compute metrics for negotiation tasks. Evaluates goal achievement, fairness, and strategy effectiveness. Returns:

Final evaluation metrics for negotiation tasks.

Compute metrics for paraphrasing tasks. Combines semantic similarity with diversity from original text. Returns:

Final evaluation metrics for paraphrasing tasks.

Compute metrics for planning tasks. Evaluates plan completeness, feasibility, and structure. Returns: float: Scor

Final evaluation metrics for planning tasks.

Compute metrics for process automation tasks. Evaluates workflow efficiency and completeness. Returns: float: Sco

Final evaluation metrics for process automation tasks.

Evaluates QA predictions using BERT Score. Args: example: The example containing the gold answer pre

Evaluates QA predictions using BERT Score. Args: example: The example containing the gold answer pre

Compute metrics for reasoning tasks. Evaluates logical consistency and step-by-step explanation. Returns: float:

Final evaluation metrics for reasoning tasks.

Compute metrics for recommendation tasks. Evaluates recommendation relevance and personalization. Returns: float:

Final evaluation metrics for recommendation tasks.

Compute metrics for Summarization tasks. Returns average of ROUGE-1, ROUGE-2, and ROUGE-L F1 scores. Returns: flo

Compute metrics for Summarization tasks. Returns average of ROUGE-1, ROUGE-2, and ROUGE-L F1 scores. Returns: flo

Compute metrics for tool use tasks. Evaluates appropriate tool selection and usage. Returns: float: Score between

Final evaluation metrics for tool use tasks.

Evaluates translation predictions using BERT Score with automatic language detection.

Evaluates translation predictions using BERT Score with automatic language detection.

Validate all configuration parameters after population. This validation ensures that all required fields are present and correctly ty

Add a new feedback to the session.

Analyze feedback and provide insights.

BERTScore wrapper that suppresses all stderr output.

Configure DSPy with the given language model

Convert dataset slice to list of dictionaries.