Preprocess input into messages format and calculate max output length. Args: input: Input prompt as string or PromptList

Process completions with a test case. Args: test_case (dict): A test case containing prompt and stop tokens. completi

Process the response to needed format. Args: datum(dict): inputs. Returns: dict: Processed response data sam

Convert the prompts to a API-style prompts, given an updated role_dict. Args: prompts (Union[List, str]): The prompts to

Convert the prompts to a string, given an updated role_dict. Args: prompts (Union[List, str, dict]): The prompt(s) to be converte

Convert a role prompt to a string, given an updated role_dict. Args: role_prompt (Dict): The role prompt to be converted.

Update the default role dict with the given prompt(s).

Update the default role dict with the given prompts.

Based on ERRANT's alignment 基于改进的动态规划算法，为原句子的每个字打上编辑标签，以便使它能够成功转换为目标句子。 编辑操作类别： 1) M：Match，即KEEP，即当前字保持不变 2)

Calculate maximum possible self-alignment score

Calculate metrics for molecular formula. Here we use element_match (equals to exact_match used in our paper) by default, which compares the atom

Check if the power expression can be computed. Parameters: expr (sympy expression): The expression to check. Returns:

Chat with agent.

判断一个单词是否全部由中文组成 :param word: :return:

Check position bias for judgellm's judgement. Args: judged_answers: The successfully extracted judgement. references: The referen

Check whether the answer solved the query.

Helper function to clean and normalize string data. It strips leading and trailing whitespace and replaces multiple whitespace characters wit

Helper function to clean and normalize string data. It strips leading and trailing whitespace and replaces multiple whitespace characters wit

Clean the units in the number.

Compute accuracy for predictions against test datasets.

Default parameters used in the paper

partially initialize a bunch of frames, for now only supports one dimensional Args: shape (): the shape of the f

Download object at the given URL to a local path. Modified from https://pytorch.org/docs/stable/hub.html#torch.hub.download_url_to_file

Estimates pass@k of each problem and returns them in an array.

Estimates pass@k of each problem and returns them in an array.

Estimates pass@k of each problem and returns them in an array.

(默认 expression1 为 Ground_Truth) 函数: 判断两个表达式是否在数学意义上等价 步骤1: 提取表达式, 防止有的模型会给出"x=1"而不是"1" 步骤2: 使用 sympy 库进行等价判断.

Computes the extended tree edit distance between trees A and B with extended-zss algorithm Args: A(Node): Root node of tree 1 B(No