Get perplexity scores given a list of inputs. Args: inputs (List[str]): A list of strings. mask_length (Optional[List

Generate the ReAct format prompt. Args: chat_history (List[Dict]): The history log in previous runs. inner_step (List

Get the paths to the output files. Every file should exist if the task succeeds. Args: file_extension (str): The file ext

Safely formats a string with the given keyword arguments. If a keyword is not found in the string, it will be ignored. Args: input_st

Summarize the results of the tasks. Args: status (list[tuple[str, int]]): A list of (task name, exit code).

Encode the raw template given in the config into a str or a PromptList. Args: prompt_template (List[Dict]] or str): The r

A helper function that builds object with MMEngine's new config.

Converts a sequence of [tokens|ids] using the vocab.

Extract the role prediction from the full prediction string. The role prediction may be the substring between the begin and end string. Args:

Format the raw data into the format that can be evaluated. Args: raw_data (str): The raw data. start_character (str, optional): T

Args: model: Mask predictor. prompt: A tensor of shape (1, L). steps: Sampling steps, less than or equal to gen_length.

Generate results given a list of inputs. Args: inputs (List[str]): A list of strings. max_out_len (int): The maximum

Generate in-context example based on the provided :obj:`entry` data. Args: entry (:obj:`Dict`): A piece of data to be used for ge

Generate corpus for input field. Args: dataset (:obj:`Dataset` or :obj:`DatasetDict`): A :obj:`datasets.Dataset`

Get the labels of the dataset, especially useful for ppl inferencer. If `ice_template` is provided, the labels will be the keys of the

Get perplexity scores given a list of inputs. Args: inputs (List[str]): A list of strings. mask_length (Optional[List

Get perplexity given a list of templates. Args: templates (List[PromptType]): A list of templates. mask_length (List[

Get a token from the bucket.

Load the dataset using shared .

Parse the intermidate prompt template, and wrap it with meta template if applicable. When the meta template is set and the input is a

Complex print function. We often use it as print_for_gpt(pad_space=1, pad_char='> ') However, more complex arguments can be used to

>>> c = CharFuncs('data/char_meta.txt') >>> c.pronunciation_similarity('牛', '午') 0.27999999999999997 >>> c.pronunciat

Convert ratings from the natural scale to the Elo rating scale with an anchored baseline.

>>> c = CharFuncs('data/char_meta.txt') >>> c.shape_similarity('牛', '午') 0.8571428571428572 >>> c.shape_similarity('田

We use APTED algorithm proposed by M. Pawlik and N. Augsten github link: https://github.com/DatabaseGroup/apted

Preprocess input into messages format and calculate max output length. Args: input: Input prompt as string or PromptList

Process the evaluation results. Args: outputs (list): List of evaluation results for each test case. prompts (list): L

Calculate the size of a subtree based on its total insertion cost. The function computes the size of a subtree by summing up the insertion

Converts `text` to Unicode (if it's not already), assuming utf-8 input.

Calculates 1 - comb(n - c, k) / comb(n, k).

Find the first available file with given extensions.

Find first valid option for text.