Base class for model's outputs, with potential hidden states and attentions. Args: last_hidden_state (:obj:`torch.FloatTensor` o

A single training/test example for simple sequence classification. Args: guid: Unique id for the example. text_a: string. Th

1D-convolutional layer as defined by Radford et al. for OpenAI GPT (and also used in GPT-2). Basically works like a linear layer but the wei

`Memory` NamedTuple have a single field `bytes` and you can get a human readable str of the number of mega bytes by calling `__repr__` -

Base class for model's outputs, with potential hidden states and attentions. Args: last_hidden_state (:obj:`tf.Tensor` of shape

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added betwee

Base class for masked language models outputs. Args: loss (:obj:`torch.FloatTensor` of shape :obj:`(1,)`, `optional`, returned w

Base class for outputs of sentence classification models. Args: loss (:obj:`torch.FloatTensor` of shape :obj:`(1,)`, `optional`,

Base class for outputs of question answering models. Args: loss (:obj:`torch.FloatTensor` of shape :obj:`(1,)`, `optional`, retu

Base class for outputs of token classification models. Args: loss (:obj:`torch.FloatTensor` of shape :obj:`(1,)`, `optional`, re

Holds the output of the :meth:`~transformers.tokenization_utils_base.PreTrainedTokenizerBase.encode_plus` and :meth:`~transformers.tokenizati

Base class for outputs of multiple choice models. Args: loss (:obj:`torch.FloatTensor` of shape `(1,)`, `optional`, returned whe

Base class for outputs of question answering models. Args: loss (:obj:`tf.Tensor` of shape :obj:`(1,)`, `optional`, returned whe

r""" Construct shared token embeddings. The weights of the embedding layer is usually shared with the weights of the linear decoder when doin

Base class for model's outputs that also contains a pooling of the last hidden states. Args: last_hidden_state (:obj:`torch.Floa

r""" Compute a single vector summary of a sequence hidden states. Args: config (:class:`~transformers.PretrainedConfig`):

1D-convolutional layer as defined by Radford et al. for OpenAI GPT (and also used in GPT-2). Basically works like a linear layer but the wei

Base class for masked language models outputs. Args: loss (:obj:`tf.Tensor` of shape :obj:`(1,)`, `optional`, returned when :obj

Base class for outputs of multiple choice models. Args: loss (:obj:`tf.Tensor` of shape `(1,)`, `optional`, returned when :obj:`

Base class for outputs of sentence classification models. Args: loss (:obj:`tf.Tensor` of shape :obj:`(1,)`, `optional`, returne

Base class for outputs of token classification models. Args: loss (:obj:`tf.Tensor` of shape :obj:`(1,)`, `optional`, returned w

Base class for model's outputs that may also contain a past key/values (to speed up sequential decoding). Args: last_hidden_stat

https://arxiv.org/pdf/2004.02984.pdf

Compute a single vector summary of a sequence hidden states. Args: config (:class:`~transformers.PretrainedConfig`): The

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added betwee

Implements Adam algorithm with weight decay fix as introduced in `Decoupled Weight Decay Regularization <https://arxiv.org/abs/1711.05101>`__

Constructs a BasicTokenizer that will run basic tokenization (punctuation splitting, lower casing, etc.). Args: do_lower_case (:obj:

Base class for causal language model (or autoregressive) outputs. Args: loss (:obj:`torch.FloatTensor` of shape :obj:`(1,)`, `op

This class copied code from :class:`~transformers.RobertaModel` and overwrote standard self-attention with longformer self-attention to provi

Base class for model's outputs that also contains a pooling of the last hidden states. Args: last_hidden_state (:obj:`tf.Tensor`

Base model with decoder

Base class for causal language model (or autoregressive) outputs. Args: loss (:obj:`torch.FloatTensor` of shape :obj:`(1,)`, `op

Evaluation output (always contains labels), to be used to compute metrics. Parameters: predictions (:obj:`np.ndarray`): Predictions

r"""A placeholder identity operator that is argument-insensitive.

Base class for sequence-to-sequence language models outputs. Args: loss (:obj:`torch.FloatTensor` of shape :obj:`(1,)`, `optiona

A single training/test example for the Squad dataset, as loaded from disk. Args: qas_id: The example's unique identifier que

this class wraps a the TFSharedEmbeddingTokens layer into a python 'no-keras-layer' class to avoid problem with weight restoring. Also it mak

Multi-headed attention from 'Attention Is All You Need' paper

Multi-headed attention from 'Attention Is All You Need' paper

r""" :class:`~transformers.DPRConfig` is the configuration class to store the configuration of a `DPRModel`. This is the configuration cl

Data collator used for language modeling. - collates batches of tensors, honoring their tokenizer's pad_token - preprocesses batches for

Data collator that will dynamically pad the inputs received. Args: tokenizer (:class:`~transformers.PreTrainedTokenizer` or :class:`

`MemoryState` are namedtuples listing frame + CPU/GPU memory with the following fields: - `frame` (`Frame`): the current frame (see above) -

Raised by a :class:`~transformers.Pipeline` when handling __call__. Args: task (:obj:`str`): The task of the pipeline. model

Base class for model encoder's outputs that also contains : pre-computed hidden states that can speed up sequential decoding. Args:

Base model without decoder

TrainingArguments is the subset of the arguments we use in our example scripts **which relate to the training loop itself**. Using :clas

Possible values for the ``truncation`` argument in :meth:`PreTrainedTokenizerBase.__call__`. Useful for tab-completion in an IDE.

AdaFactor pytorch implementation can be used as a drop in replacement for Adam original fairseq code: https://github.com/pytorch/fairseq/blob

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added betwee

The best run found by an hyperparameter search (see :class:`~transformers.Trainer.hyperparameter_search`). Parameters: run_id (:obj:

Character span in the original string. Args: start (:obj:`int`): Index of the first character in the original string. end (: