get command token corresponding to `name`

Return an instance of this string that is marked as shortenable

encode text using text tokenizer and shift Id values for command tokens

given a generator of metrics for each of the data points X_i, write the metrics, text, and labels to a csv file

convert Ids to tokens accounting for command and type tokens, tokens are joined and returned as a string.

check if the filepath for a text tokenizer exists

Encode an input example using this pattern-verbalizer pair. :param example: the input example to encode :param priming: whet

Start the timer.

Ensure that numerator is divisible by the denominator and return the division value.

Set random seed for reproducability.

Stop the timer.

Get the model parallel group the caller rank belongs to.

Remove the final punctuation mark

Convert to numpy and return a sample consumed by the batch producer.

Return world size for the model parallel group.

convert Id to token accounting for command and type tokens

Lowercase the first character

Return my rank for the model parallel group.

Loads weights from src to dst via in place copy. src is a huggingface gpt2model, while dst is one of our models. dst2src=True loads param

Add a new hypothesis to the list.

Get the token ids corresponding to a verbalization :param word: the verbalization :param tokenizer: the tokenizer to use :param forc

Fork the cuda rng state, perform operations, and exit with the original state.

Instantiate a PreTrainedBertModel from a pre-trained model file. Download and cache the pre-trained model file if needed.

Get cuda rng tracker.

If no closure is supplied, :attr:`step` should be called after ``fp16_optimizer_obj.backward(loss)``. :attr:`step` updates t

Truncate two sequences of text to a predefined total maximum length

Sets the random number generator state of the current GPU. Argumentss: new_state (torch.ByteTensor): The desired state This function

Check if we've already made a lazy version of this file for the `data_type` field.

Gets directory path where lazy files are stored.

Initialize torch.distributed.

Log a group of timers.

Save a model checkpoint.

Return all verbalizations for a given label. :param label: the label :return: the list of verbalizations

Initialize affine weight for model parallel. Build the master weight on all processes and scatter the relevant chunk.

Transpose a 3D tensor [b, s, np*hn] into a 4D tensor with size [b, np, s, hn].

Data loader. Note that batch-size is the local (per GPU) batch-size.

Helper function to evaluate and dump results on screen.

Main finetune function used across all tasks.

Get rng states. Copy the dictionary so we have direct pointers to the states, not just a pointer to the dictionary.

Load a model checkpoint.

Lower text and remove punctuation, articles and extra whitespace.

Split a tensor along its last dimension. Arguments: tensor: input tensor. num_partitions: number of partitions to split the tensor

Transpose a 3D tensor [b, s, np*hn] into a 4D tensor with size [b, np, s, hn].

:attr:`backward` performs the following conceptual steps: 1. fp32_loss = loss.float() (see first Note below) 2. scaled_loss

Calculate the elapsed time.

Parse all the args.

get_batch subdivides the source data into chunks of length args.seq_length. If source is equal to the example output of the data loading exam

Initialize torch.distributed.

Forward step.