Replaces all instances of 'src' in the PromptList with 'dst'. Args: src (str): The string to be replaced. dst (Prompt

Replaces all instances of 'src' in the PromptList with 'dst'. Args: src (str): The string to be replaced. dst (Prompt

return dataset id when getting data from ModelScope/HuggingFace repo, otherwise just return local path as is. Args: dataset_id (str):

see torch.sum Args: dim (): keepdim (): Returns:

Encode prompt to tokens. Not necessary for most cases. Args: prompt (str): Input string. Returns: torch.Tens

Update pbar counter when callback.

Generate model abbreviation from the model's confg.

Returns dataset abbreviation from the dataset's confg.

Get the logger for OpenCompass. Args: log_level (str): The log level. Default: 'INFO'. Choices are 'DEBUG', 'INFO', 'WARNING'

Decode tokens to text. Not necessary for most cases. Args: tokens (torch.Tensor): Input tokens. Returns: str

See Tensor.view Args: *args (): Returns:

Ensure simultaneous emptying of stdout and stderr when concurrent resources are available. When employing multiprocessing with standa

see torch.squeeze Args: dim (): Returns:

Wait till the next query can be sent. Applicable in both single-thread and multi-thread environments.

Get lengths given a list of templates. Args: templates (Union[List[str], str]): Input template(s). mode (str): Parsin

Generate the in-context example for one test example. If `ice_template` is an instance of `PromptTemplate`, the `ice_separator` and `i

Get out put path. Args: cfg (ConfigDict): The running config. time_str (str): Current time.

Release concurrent resources if acquired. This behavior will fall back to do nothing if there are no concurrent resources.

Post a message to Lark. When title is None, message must be a str. otherwise msg can be in rich text format (see https://open

Acquire concurrent resources if exists. This behavior will fall back to wait with query_per_second if there are no concurrent resourc

Extract judgements (scores) and references. Args: result (ConfigDict): Dataset config. filename (str): Model path in results dir.

Parse a prompt template, and wrap it with meta template if applicable. Args: prompt_template (List[PromptType]): A prompt

Generate an item based on the provided :obj:`entry` data, as well as optional output field and ice field tokens. Warning:

Generate the prompt for one test example in generative evaluation with `prompt_template`. If `prompt_template` is not provided, the `i

Filter the repeated error message. Args: record (LogRecord): The log record. Returns: bool: Whether to outpu

Extract judgements (scores) and references. Args: dataset (ConfigDict): Dataset config. subdir_path (str): Model path in results

Construct a frame Args: trans: the absolute position in the bigger frame rots: the rotation of the frame

Generate completion from a list of templates. Args: templates (List[PromptType]): A list of templates. max_out_len (i

Get the command template for the task. Args: cfg_path (str): The path to the config file of the task. template (str):

Returns task abbreviation from the task's confg.

Processes a list of dictionaries to compute markdown element counts. Args: data (list): Input data, either a list of dictionaries.

Return a dataloader of the input data list.

Get the hash of the prompt configuration. Args: dataset_cfg (ConfigDict or list[ConfigDict]): The dataset configuration.

Parse the package dependencies listed in a requirements file but strips specific versioning information. Args: fname (str): path to r

Dump the result to a json file.

Calculate toxic scores for each prediction. Args: predictions (List): List of predictions of each sample. Returns:

Normalize a final answer to a quantitative reasoning question.

Find the first available file with given extensions.

Generate the prompt for one test example in perpelxity evaluation with `prompt_template`. If `prompt_template` is not provided, the `i

Load data from a JSON or JSONL file.

Lower text and remove punctuation, articles and extra whitespace.

Lower text and remove punctuation, extra whitespace.

>>> edit_distance('abcde', 'avbcude') 2 >>> edit_distance(['至', '刂'], ['亻', '至', '刂']) 1 >>> edit_distance('fang', 'qwe') 4

Extracts text enclosed in double brackets [[ ]]. Args: text (str): The text to extract from. clean_level (str): The l