Return an instance of this string that is marked as shortenable

Load a pretrained wrapper from a given path.

Load a config from a file.

Get the token ids corresponding to a verbalization :param word: the verbalization :param tokenizer: the tokenizer to use :param force

Creates examples for the training and dev sets.

Add a new input example to this list. :param example: the example to add :returns: `true` iff the example was actually added

Encode an input example using this pattern-verbalizer pair. :param example: the input example to encode :param priming: whet

Equally divide N examples among i buckets. For example, `eq_div(12,3) = [4,4,4]`.

Load examples for a given task.

Remove the final punctuation mark

Return all verbalizations for a given label. :param label: the label :return: the list of verbalizations

Compute the exact match (EM) for a sequence of predictions and actual labels

Lowercase the first character

Save this config to a file.

Save an array of logits to a file

Save a sequence of predictions to a file

Create a new LogitsList. :param score: the corresponding PET model's score on the training set :param logits: the list of log

Create a new task helper. :param wrapper: The wrapper for the language model being used.

Shuffle a list of examples and restrict it to a given maximum size. :param examples: the examples to shuffle and restrict :param num_exa

Add special features from the ``meta`` dictionary of a sequence of features to the corresponding dictionary :param features: the sequ

Add special features to the ``meta`` dictionary of a feature set :param input_example: the input example considered :param in

Build a new model instance.

Custom implementation of the eval step for this task. :param batch: a batch of examples :return: a tensor of logits

Convert the given example into a set of input features

Given an input example, apply a pattern to obtain two text sequences (text_a and text_b) containing exactly one mask token (or one co

Initialize a new model from the given config.

Load a list from a file

Load Megatron_lm in fp16. A Tesla V100 is enough for inference. I haven't implement the parallel method for fine-tuning. You can refer to

Load the model, training and evaluation configs for PET from the given command line arguments.

Set RNG seeds for python's `random` module, numpy and torch

Serialize this instance to a Python dictionary.

Serialize this instance to a Python dictionary.

Serialize this instance to a JSON string.

Serialize this instance to a JSON string.

Train the underlying language model. :param train_data: the training examples to use :param per_gpu_train_batch_size: the nu

Train and evaluate a new PET model for a given task. :param model_config: the model configuration for each model corresponding to an individ

Train a single model. :param model: the model to train :param train_data: the training examples to use :param config: the training co

Custom implementation of the train step for this task. :param batch: a batch of examples :return: a scalar loss tensor

Trim a sequence of input ids by removing all padding tokens and keeping at most a specific number of mask tokens. :param input_ids: the seque

Truncate two sequences of text to a predefined total maximum length