Initialize and prunes weights if needed.

Converts a sequence of ids (integer) in a string, using the tokenizer and vocabulary with options to remove special tokens and clean

projection

Reads a tab separated value file.

r""" Instantiates one of the base model classes of the library from a pre-trained model configuration. The model class to instantiate

Converts a string in a sequence of ids (integer), using the tokenizer and vocabulary. Same as doing ``self.convert_tokens_to_ids(sel

Reads a tab separated value file.

Reads a tab separated value file.

Reads a tab separated value file.

Reads a tab separated value file.

Reads a tab separated value file.

Prune a linear layer (a model parameters) to keep only entries in index. Return the pruned layer as a new layer with requires_grad=True.

Reads a tab separated value file.

Reads a tab separated value file.

Creates a `truncated_normal_initializer` with the given range.

Tie or clone module weights depending of weither we are using TorchScript or not

Converts a single token, or a sequence of tokens, (str/unicode) in a single integer id (resp. a sequence of ids), using the vocabulary.

Reads a tab separated value file.

Reads a tab separated value file.

Reads a tab separated value file.

Reads a tab separated value file.

Reads a tab separated value file.

Reads a tab separated value file.

Reads a tab separated value file.

Reads a tab separated value file.

Reads a tab separated value file.

Project hidden states to a specific head with a 4D-shape.

Reads a tab separated value file.

Creates a `truncated_normal_initializer` with the given range.

Creates a `truncated_normal_initializer` with the given range.

Creates a `truncated_normal_initializer` with the given range.

Creates a `truncated_normal_initializer` with the given range.

Creates a `truncated_normal_initializer` with the given range.

Creates a `truncated_normal_initializer` with the given range.

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Reads a tab separated value file.

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Creates or gets the iterations_per_loop variable. In TPUEstimator, the user provided computation, the model_fn, is wrapped inside a tf.while_loop

Build a resized Embedding Module from a provided token Embedding Module. Increasing the size will add newly initialized vectors at the en

Conv1D layer as defined by Radford et al. for OpenAI GPT (and also used in GPT-2) Basically works like a Linear layer but the weights are

Gets `features` and `labels`.

Basic Tokenization of a piece of text. Split on "white spaces" only, for sub-word tokenization, see WordPieceTokenizer. Args: