Returns a list of the lemmatized text of each token. Returns None if this annotation was not included.

Customized load.

Customized load.

Load the state dicts of each of the models

Customized load.

Loads a state_dict created by an earlier call to state_dict(). If ``fp16_optimizer_instance`` was constructed from some ``init_optimi

Load T5 models from saved checkpoints

Returns a 3-dimensional (3-D) attention mask :param source_block: 1-D array :param target_block: 1-D array

Buld dataloader given an input dataset.

Check if mips group is initialized.

Initialize model parallel cuda seed. This function should be called after the model parallel is initialized. Also, no torch.cuda.manual_seed

Check if model and data parallel groups are initialized.

Build the model.

Convert model to half precision in a batchnorm-safe way. Retained for legacy purposes. It is recommended to use FP16Model.

Returns a list of all ngrams from length 1 to n. Args: n: upper limit of ngram length uncased: lower cases text

Return number of elements in use.

Returns a list of [start, end) character offsets of each token.

Returns a list of part-of-speech tags of each token. Returns None if this annotation was not included.

Creates a list of FP32 master parameters for a given model, as in `Training Neural Networks with Mixed Precision: Real Examples`_. Args

Print memory usage average over time. We would like this value to be as high as possible.

Print min, max, and norm of all parameters.

Returns text encoded in a way suitable for print or `tf.logging`.

Implement abstract method.

Reset the buffer start index to the beginning of the buffer.

Reset the memory used for checkpointing.

Delete existing index and create anew

Delete existing index and create anew

Get the top-k blocks by the index distance metric. if False: return [num_queries x k] array of distances, and another for indices

Returns a dict containing the current state of this :class:`FP16_Optimizer` instance. This dict contains attributes of :class:`FP16_O

For easy load when model is combined with other heads, add an extra key.

For easy load.

For easy load when model is combined with other heads, add an extra key.

For easy load when model is combined with other heads, add an extra key.

Tokenizes a piece of text.

Tokenizes a piece of text into its word pieces. This uses a greedy longest-match-first algorithm to perform tokenization using the gi