Generate prompt based on :obj:`entry` data, :obj:`ice` in-context example, and the corresponding :obj:`label`. Args: ent

Get the path to the log file. Args: file_extension (str): The file extension of the log file. Default: 'json'.

Parse a prompt template, and wrap it with meta template if applicable. Args: prompt_template (List[PromptType]): A prompt

This function converts ids string to a string that can be used as a tree input to APTED. Any Error raised by this function implies that the i

The function calculates the numerical results for each group based on the configuration in summary_groups, and updates the contents of

The function reads the numerical results of evaluations from the output folder based on the configuration file, and ultimately returns

Calculates 1 - comb(n - c, k) / comb(n, k).

Generate the ReAct format prompt. Args: chat_history (List[Dict]): The history log in previous runs. inner_step (List

Safely formats a string with the given keyword arguments. If a keyword is not found in the string, it will be ignored. Args: input_st

Summarize the results of the tasks. Args: status (list[tuple[str, int]]): A list of (task name, exit code).

从模型的原始输出中提取预测结果（Yes或No）。 此函数会查找并返回跟在The answer is后面的Yes或者No， 或从文本中识别常见的Yes/No表达方式。

Encode the raw template given in the config into a str or a PromptList. Args: prompt_template (List[Dict]] or str): The r

Create a rigid 4x4 transformation matrix from two axes and transl.

A helper function that builds object with MMEngine's new config.

Converts a sequence of [tokens|ids] using the vocab.

Extract the role prediction from the full prediction string. The role prediction may be the substring between the begin and end string. Args:

Format the raw data into the format that can be evaluated. Args: raw_data (str): The raw data. start_character (str, optional): T

Generate results given a list of inputs. Args: inputs (List[str]): A list of strings. max_out_len (int): The maximum

Generate in-context example based on the provided :obj:`entry` data. Args: entry (:obj:`Dict`): A piece of data to be used for ge

Generate corpus for input field. Args: dataset (:obj:`Dataset` or :obj:`DatasetDict`): A :obj:`datasets.Dataset`

Get the labels of the dataset, especially useful for ppl inferencer. If `ice_template` is provided, the labels will be the keys of the

Get the paths to the output files. Every file should exist if the task succeeds. Args: file_extension (str): The file ext

Get perplexity scores given a list of inputs. Args: inputs (List[str]): A list of strings. mask_length (Optional[List

Get perplexity given a list of templates. Args: templates (List[PromptType]): A list of templates. mask_length (List[

Get a token from the bucket.

Load the dataset using shared .

Lower text and remove punctuation, articles and extra whitespace.

Parse the intermidate prompt template, and wrap it with meta template if applicable. When the meta template is set and the input is a

Complex print function. We often use it as print_for_gpt(pad_space=1, pad_char='> ') However, more complex arguments can be used to

>>> c = CharFuncs('data/char_meta.txt') >>> c.pronunciation_similarity('牛', '午') 0.27999999999999997 >>> c.pronunciat

Convert ratings from the natural scale to the Elo rating scale with an anchored baseline.

>>> c = CharFuncs('data/char_meta.txt') >>> c.shape_similarity('牛', '午') 0.8571428571428572 >>> c.shape_similarity('田

We use APTED algorithm proposed by M. Pawlik and N. Augsten github link: https://github.com/DatabaseGroup/apted

Pass the input through the linear and perform reshaping for the multi-headed attention, then transform the points by the transformati

Pass the input through linear and then perform reshaping for the multi-headed attention Args: linear: the linear

Process the evaluation results. Args: outputs (list): List of evaluation results for each test case. prompts (list): L

Calculate the size of a subtree based on its total insertion cost. The function computes the size of a subtree by summing up the insertion