Get a collection of `InputExample`s for the train set.

Initialize the weights.

Initialize model data parallel groups. Arguments: model_parallel_size: number of GPUs used to parallelize model. Let's say we h

When running with :class:`FP16_Optimizer`, ``.grad`` attributes of a model's fp16 leaves should not be regarded as truthful,

Load a set of input examples from a file

Loads a state_dict created by an earlier call to state_dict(). If ``fp16_optimizer_instance`` was constructed from some ``init_optim

Load tf checkpoints in a pytorch model

Main program.

function to create datasets+tokenizers for common options

Return the underlying LM's mask token

Return the underlying LM's mask token

Return the underlying LM's mask id

Return the underlying LM's mask id

Return the maximum number of verbalizers across all labels

Privde metrics callback function.

Provide metrics callback function.

Privde metrics callback function.

Initialize model parallel cuda seed. This function should be called after the model parallel is initialized. Also, no torch.cuda.manual_seed

Check if model and data parallel groups are initialized.

Loads weights from `oai` to `our` via in place copy. `oai` is a huggingface gpt2model, while `our` is one of our models. dst2src=True loa

Calculate correct over total answers and return prediction if the `output_predictions` is true.

Compute the exact match (EM) for a sequence of predictions and actual labels

Convert model to half precision in a batchnorm-safe way. Retained for legacy purposes. It is recommended to use FP16Model.