Core relative positional attention operations.

Core relative positional attention operations.

This function wraps another function into its own separated process. In order to ensure accurate memory measurements it is important that t

Set the vebosity level for the 🤗 Transformers's root logger. Args: verbosity (:obj:`int`): Logging level, e.g., one of:

Shift input ids one token to the right, and wrap the last non pad token (usually <eos>).

Basic Tokenization of a piece of text. Split on "white spaces" only, for sub-word tokenization, see WordPieceTokenizer. Args:

Converts a string in a sequence of tokens, using the tokenizer. Split in words for word-based vocabulary or sub-words for sub-word-b

Runs basic whitespace cleaning and splitting on a piece of text.

Creates examples for the training, dev and test sets.

Creates examples for the training, dev and test sets.

Creates examples for the training, dev and test sets.

Creates examples for the training, dev and test sets.

Creates examples for the training, dev and test sets.

Creates examples for the training, dev and test sets.

Creates examples for the training, dev and test sets.

Creates examples for the training, dev and test sets.

force one of token_ids to be generated by setting prob of all other tokens to 0 (logprob=-float("inf"))

Build a resized Embedding Module from a provided token Embedding Module. Increasing the size will add newly initialized vectors at th

HP search setup code

Checks whether `char` is a control character.

Checks whether `char` is a punctuation character.

Checks whether `char` is a whitespace character.

Add a dictionary of special tokens (eos, pad, cls, etc.) to the encoder and link them to class attributes. If special tokens are NOT

Tokenize and prepare for the model a list of sequences or a list of pairs of sequences. .. warning:: This method is depr

Utility to convert a number of bytes (int) into a number of mega bytes (int)

How the loss is computed by Trainer. By default, all models return the loss in the first element. Subclass and override for custom b

Converts a sequence of tokens (string) in a single string.

Setup the optimizer and the learning rate scheduler. We provide a reasonable default that works well. If you want to use something e

The device used by this process.

Tokenize and prepare for the model a sequence or a pair of sequences. .. warning:: This method is deprecated, ``__call__

For models that inherit from :class:`~transformers.PretrainedModel`, uses that method to compute the number of floating point operati

Constructs a `ModelCard` from a Python dictionary of parameters.

Returns the model's input embeddings. Returns: :obj:`nn.Module`: A torch module mapping vocabulary to hidden states.

Create a schedule with a learning rate that decreases linearly from the initial lr set in the optimizer to 0, after a warmup period during wh

Returns the the linear layer that produces question answering logits. Returns: :obj:`nn.Module`: A torch module mapping

Returns the the linear layer that produces question answering logits Returns: :obj:`nn.Module`: A torch module mapping t

The accumulated gradients on the current replica.

Turns 1->0, 0->1, False->True, True-> False

Whether or not this process is the local (e.g., on one machine if training in a distributed fashion on several machines) main process

Whether or not this process is the local (e.g., on one machine if training in a distributed fashion on several machines) main process

Log :obj:`logs` on the various objects watching training. Subclass and override this method to inject custom behavior. Args

Prepare masked tokens inputs/labels/attention_mask for masked language modeling: 80% MASK, 10% random, 10% original. N-gram not appli

Get number of (optionally, trainable or non-embeddings) parameters in the module. Args: only_trainable (:obj:`bool`, `op

pickle.dump(obj, path)

Utility factory method to build a :class:`~transformers.Pipeline`. Pipelines are made of: - A :doc:`tokenizer <tokenizer>` in charg

Post-attention processing.

Post-attention processing.

Will save the model, so you can reload it using :obj:`from_pretrained()`. Will only save from the world_master process (unless in TP

Reset the special tokens to the source lang setting. No prefix and suffix=[eos, cur_lang_code].