The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added betwee

Computes and stores the average and current value

BatchEncoding hold the output of the encode and batch_encode methods (tokens, attention_masks, etc). This class is derived from a python Dict

r"""A placeholder identity operator that is argument-insensitive.

CfgNode represents an internal node in the configuration tree. It's a simple dict-like container that allows for attribute-based access to ke

Character span in the original string Args: start: index of the first character in the original string end: index of

Runs basic tokenization (punctuation splitting, lower casing, etc.).

Construct the embeddings from word, position and token_type embeddings.

Multilayer perceptron.

Image to Patch Embedding Args: patch_size (int): Patch token size. Default: 4. in_chans (int): Number of input image channels. D

Compute SQuAD 2.0 answer class from classification and start tokens hidden states.

Compute SQuAD end_logits from sequence hidden states and start token hidden state.

Compute SQuAD start_logits from sequence hidden states.

r""" Base class for all configuration classes. Handles a few parameters common to all models' configurations as well as methods for loading/do

stream object that redirects writes to a logger instance.

Swin Transformer Block. Args: dim (int): Number of input channels. num_heads (int): Number of attention heads. window_si

Token span in an encoded string (list of tokens) Args: start: index of the first token in the span end: index of the

Window based multi-head self attention (W-MSA) module with relative position bias. It supports both of shifted and non-shifted window. Args:

Runs WordPiece tokenization.

r""" This is the configuration class to store the configuration of a :class:`~transformers.BertModel`. It is used to instantiate an BE

An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models.

r""" Constructs a BERT tokenizer. Based on WordPiece. This tokenizer inherits from :class:`~transformers.PreTrainedTokenizer` which contains

Enum with more explicit error message for missing values.

A class contraining all of the functions supporting generation, to be used as a mixin in PreTrainedModel.

A few utilities for torch.nn.Modules, to be used as a mixin.

Patch Merging Layer Args: dim (int): Number of input channels. norm_layer (nn.Module, optional): Normalization layer. Default:

r""" Base class for all models. :class:`~transformers.PreTrainedModel` takes care of storing the configuration of the models and handles meth

Base class for all slow tokenizers. Handle all the shared methods for tokenization and special tokens as well as methods downloading/caching

Base class for slow and fast tokenizers. Handle shared (mostly boiler plate) methods for slow and fast tokenizers.

r""" A SQuAD head inspired by XLNet. Parameters: config (:class:`~transformers.XLNetConfig`): Model configuration class with all the para

r""" Compute a single vector summary of a sequence hidden states according to various possibilities: Args of the config class: sum

SpecialTokensMixin is derived by ``PreTrainedTokenizer`` and ``PreTrainedTokenizerFast`` and handles specific behaviors related to special to

Descriptor that mimics @property but caches output in member variable. From tensorflow_datasets Built-in in functools from Python 3.8.