A single training/test example for simple sequence classification. Args: guid: Unique id for the example. text_a: string. Th

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added betwee

This subclass of `argparse.ArgumentParser` uses type hints on dataclasses to generate arguments. The class is designed to play well with

Implements Adam algorithm with weight decay fix. Parameters: lr (float): learning rate. Default 1e-3. betas (tuple of 2 floats):

Runs basic tokenization (punctuation splitting, lower casing, etc.).

r""" This is the configuration class to store the configuration of a :class:`~transformers.ReformerModel`. It is used to instantiate a

https://arxiv.org/pdf/2004.02984.pdf

Construct shared token embeddings.

r""" Configuration class for Bart. Parameters are renamed from the fairseq implementation

Trainer is a simple but feature-complete training and eval loop for PyTorch, optimized for Transformers.

BatchEncoding hold the output of the encode and batch_encode methods (tokens, attention_masks, etc). This class is derived from a python Dict

r""" :class:`~transformers.EncoderDecoder` is a generic model class that will be instantiated as a transformer architecture with one o

This class overrides :class:`~transformers.RobertaModel` to provide the ability to process long sequences following the selfattention approac

This class overrides :class:`~transformers.BertModel`. Please check the superclass for the appropriate documentation alongside usage examples

r""" Compute a single vector summary of a sequence hidden states according to various possibilities: Args of the config class: sum

Same as TFBertMainLayer but uses TFRobertaEmbeddings.

r""" Compute a single vector summary of a sequence hidden states according to various possibilities: Args of the config class: sum

This will be superseded by a framework-agnostic approach soon.

Runs basic tokenization with MeCab morphological parser.

This module produces sinusoidal positional embeddings of any length.

r""" Outputs: `Tuple` comprising various elements depending on the configuration (config) and inputs: **last_hidden_state**: ``torch.Float

A single training/test example for multiple choice Args: example_id: Unique id for the example. question: string. The untoke

This will be superseded by a framework-agnostic approach soon.

The `MaskedBertModel` class replicates the :class:`~transformers.BertModel` class and adds specific inputs to compute the adaptive mask on th

Fully Connected layer with on the fly adaptive mask. If needed, a score matrix is created to store the importance of each associated weight.

Transformer-XL tokenizer adapted from Vocab class in https://github.com/kimiyoung/transformer-xl This tokenizer inherits from :class:`~trans

Runs WordPiece tokenization.

Constructs an XLNet tokenizer. Based on `SentencePiece <https://github.com/google/sentencepiece>`__ This tokenizer inherits from :class:`~tr

this class wraps a the TFSharedEmbeddingTokens layer into a python 'no-keras-layer' class to avoid problem with weight restoring. Also it m

Constructs an ALBERT tokenizer. Based on `SentencePiece <https://github.com/google/sentencepiece>`__ This tokenizer inherits from :class:`~t

r""" This is the configuration class to store the configuration of a :class:`~transformers.BertModel`. It is used to instantiate an BE

Data collator used for language modeling. - collates batches of tensors, honoring their tokenizer's pad_token - preprocesses batches for

This is the configuration class to store the configuration of a :class:`~transformers.GPT2Model`. It is used to instantiate an GPT-2

Arguments pertaining to what data we are going to input our model for training and eval. Using `HfArgumentParser` we can turn this class

Gradient accumulation utility. When used with a distribution strategy, the accumulator should be called in a replica context. Gradients will b

`Memory` NamedTuple have a single field `bytes` and you can get a human readable str of the number of mega bytes by calling `__repr__`

This is the configuration class to store the configuration of a :class:`~transformers.OpenAIGPTModel`. It is used to instantiate an G

r""" This is the configuration class to store the configuration of a :class:`~transformers.RobertaModel`. It is used to instantiate an

Same as BertEmbeddings with a tiny tweak for positional embeddings indexing.

A single training/test example for the Squad dataset, as loaded from disk. Args: qas_id: The example's unique identifier que

Constructs a SquadResult which can be used to evaluate a model's output on the SQuAD dataset. Args: unique_id: The unique identifier

r""" :class:`~transformers.T5Config` is the configuration class to store the configuration of a `T5Model`. Arguments:

This will be superseded by a framework-agnostic approach soon.

This is the configuration class to store the configuration of a :class:`~transformers.TransfoXLModel`. It is used to instantiate a Tr

r""" This is the configuration class to store the configuration of a :class:`~transformers.AlbertModel`. It is used to instantiate an

This is the configuration class to store the configuration of a :class:`~transformers.CTRLModel`. It is used to instantiate an CTRL m

Character span in the original string Args: start: index of the first character in the original string end: index of

Runs Character tokenziation.