An enhanced list, used for intermidate representation of a prompt.

A LogicNode is a tree primitive. It is either a deduction or a leaf fact. Leaf facts are the ones that we use in story generation (if they a

Intermidate prompt template parser, specifically for language models. Args: meta_template (Dict): The meta template for the model.

Intermidate prompt template parser, specifically for API models. Args: meta_template (Dict): The meta template for the model.

In-context Learning Prompt Template Class This class represents a template that guides the generation of prompts in the retrieval or inference

Planning Evaluation Args: dataset_path(str): File path of evaluation dataset bert_score_model(str): the bert_score model for sente

Args: template(str): Format string with keyword-only arguments. For example '{who} like {what}' pred(Any): Parsed dat

分词器

Model wrapper around OpenAI's models. Args: path (str): The name of OpenAI's model. max_seq_len (int): The maximum allowed sequen

Main datastructure used when creating a MuSR example. It's basically a standard tree with some parameters controlling the shape.

Helper class to wrap a callable into a class with a `tokenize` method as used by rouge-score.

对齐错误句子和正确句子， 使用编辑距离算法抽取编辑操作

Generation Inferencer class to directly evaluate by generation. Attributes: model (:obj:`BaseModelWrapper`, optional): The module to infe

A dataset wrapper around the evaluator inputs, designed for OpenCompass's internal use.

Zeroshot Retriever. The retriever returns empty list for all queries. Args: dataset (`BaseDataset`): Any BaseDataset instances.

Runs basic tokenization (punctuation splitting, lower casing, etc.).

A IPython executor that can execute Python scripts in a jupyter manner. Args: description (str): The description of the action. Defaults

Instruct Following Evaluation Args: dataset_path(str): File path of evaluation dataset.

A curses menu that allows the user to select one item from each list. Args: lists (list[list[str]]): A list of lists of strings, where ea

合并编辑操作，从Token-Level转换为Span-Level

Criteria to stop on the specified multi-token sequence.

OpenICL Inference Task. This task is used to run the inference process.

Planning Evaluation Args: dataset_path(str): File path of evaluation dataset name_weight(float): the weight of action_name in bert

Review Capability Evaluation Args: dataset_path(str): File path of evaluation dataset.

Response from a sampler.

A token bucket for rate limiting. Args: query_per_second (float): The rate of the token bucket.

APITemplateParser accepts OpenAI format dialog inputs.

Sample from OpenAI's chat completion API.

错误类型分类器

This object takes in command and executes it with time out.

In-conext Learning Dataset Reader Class Generate an DatasetReader instance through 'dataset'. Attributes: dataset (:obj:`Dataset` or

Result of running an evaluation (usually consisting of many samples)

Filter the repeated message. Args: name (str): name of the filter.

LMTemplateParser accepts OpenAI format dialog inputs.

Executes code locally with custom time and memory limits. Refined to distinguish between Runtime and Memory Limit errors.

Math agent evaluator for soft condition. Args: action (str): Action for catching internal prediction. Defaults to `PythonInte

Criteria to stop on the specified multi-token sequence.

Configuration for latex normalization. Each field controls a group of related normalizations: - basic_latex: Basic latex command replacements

OpenICL Inference Task. This task is used to run the inference process.

OpenICL Evaluation Task. This task is used to evaluate the metric between predictions and references.

Perspective api client for toxic evaluation. Args: key (str): Perspective API key. If set to `ENV`, find it in environment va

Result of evaluating a single sample.

Subjective Evaluation Task. This task is used to evaluate the metric between predictions and references. Args: cfg (ConfigDict):