Performs backward pass to evaluate the case and get suggestions for optimizing the SOP. Args: case (Case): The case to b

forward case to get the result and save it to the file. Args: case: the case to be forwarded solution_or_pat

Evaluates the functional correctness of a completion by running the test suite provided in the problem. :param completion_id: an optiona

Check if the new prompt is legal. This function checks whether the new prompt (given as a JSON string) is a valid update to

Checks the legality of the SOP optimization operations provided by the LLM. Args: optim_result (str): The optimization r

r"""Chunks elements by titles. Args: elements (List[Any]): List of Element objects to be chunked. chunk_type (str): T

get prompt from state depend on your role Return: system_prompt:system_prompt for agents's LLM last_prompt:last_promp

Executes the node optimization based on the given method. Note: If an error occurs during optimization, an exception is raised inste

Performs SOP optimization based on the given operation. Args: solution (Solution): The current solution. op

Achieved early stop in training Args: ave_score_list (list[float]): The score obtained for each case of the current step

r"""Generates embeddings for the given texts. Args: objs (List[str]): The texts for which to generate the embeddings.

Escapes special characters in a JSON string. In the JSON strings provided by large models, there may be special characters t

Estimates pass@k of each problem and returns them in an array.

Calculates 1 - comb(n - c, k) / comb(n, k).

Find the outermost tags in the given string. This function matches the outermost tags in the provided string using a greedy matching

Create a dictionary representing a case at a specific index. Args: idx (int): The index of the case. Returns:

Create a dictionary representing a case at a specific index, including the ground truth if available. Args: idx (int): The i

Args: start_tag (str): start_tag end_tag (str): end_tag text (str): complete sentence Returns: str: the con

Get information needed for backward and training processes. Args: keys (list): The list of keys for the required informa

Gets the order of the prompt configuration based on the availability of ground truth and evaluation score. Returns: list

case跑完了之后加载保存下来的case获取分数，会获取路径下所有的case的分数，然后计算加权平均分数 Returns:

Get the optimizer to call for the current step There are four choices for the order of optimizers: order, random, up_to_down, and do

Loads a State object from a JSON dictionary. Args: state_json (dict): The JSON dictionary containing the state data.

Determine the next node and the agent that needs action based on the current situation Return : next_node(node) : the next no

Evaluates the node and saves the result to the file. Args: case (Case): The case to be evaluated. solution (

observe the environment and return the observation Return: observation(dict)

Optimizes the configuration of each node in the solution based on the cases. Args: case_list (list[Case]): The list of c

Optimizes the agent's prompt to minimize loss. Args: case_list (list[Case]): A batch of cases. Returns:

Optimizes the configuration information of a single node. Args: case_list (list[Case]): The list of cases. n

Performs parallel backward optimization for a list of cases. Args: case_list (list[Case]): The list of cases to be optim

Calculates the loss for a list of cases in parallel. Args: case_list (list): The list of cases. parallel_max

Parallelly execute a list of partial functions pre-filled with parameters. This method runs a list of partially filled functions in

r"""Loads a file or a URL and parses its contents as unstructured data. Args: input_path (str): Path to the file or URL to be par

r"""Processes content from a file or URL, divides it into chunks by using `Unstructured IO`, and stores their embeddings in the specified

r"""Query the results. Subclasses should implement this method according to their specific needs. Args: query (str): Quer

r"""Executes a query in vector storage and compiles the retrieved results into a dictionary. Args: query (str): Query str

r"""Searches for similar vectors in the storage based on the provided query. Args: query (VectorDBQuery): The query object contai

This disables various destructive functions and prevents the generated code from interfering with the test (e.g. fork bomb, killing other pro

roll back to the last solution if the new solution is worse than the last solution If the score of the new solution on the current s

Determine the role that needs action based on the current situation Return : next_agent_name(str) : the name of the next act

Extract self.batch_size cases from the dataset. Args: sample_kind (str): The method for extracting cases from the datase

Saves the final optimization results. Args: step_save_path (str): The path to save the step information. opt

Initialize the logging configuration. This function sets up logging to both a file and the console. Log messages are saved to a file wit

return actions by current state and environment Return: action(Action)

Determine the next node based on the current situation Return : next_node(node) : the next node

Checks each prompt for validity and updates the solution if valid. Args: case_list (list[Case]): The list of cases.

Attempts to optimize the solution using the given LLM result. Args: solution (Solution): The current solution.

Updates the prompt in the solution. Args: solution (Solution): The solution to be updated. node_name (str):

Validates the optimization method list. Args: optim_method_list (list): The list of optimization methods. Retur

r"""Deletes the collection if :obj:`del_collection` is set to :obj:`True`.

Retrieve an item by its index. Args: idx (int): The index of the item to retrieve. Returns: str: The item a

Retrieve a programming prompt by its index. Args: idx (int): The index of the prompt. Returns: str:

Retrieve a question by its index. Args: idx (int): The index of the question. Returns: str: The question te

Abstract method to get an item by index. Args: idx (int): The index of the item. Returns: The item at the s

Retrieve a question by its index. Args: idx (int): The index of the question. Returns: str: The question te

Retrieve a math problem by its index. Args: idx (int): The index of the math problem. Returns: str: The mat

:param engine: The name of the search engine used :param api: Pass in a dictionary, such as {"bing":"key1", "google":"key2", ...}, of