Initialize and prunes weights if needed.

Converts a sequence of ids (integer) in a string, using the tokenizer and vocabulary with options to remove special tokens and clean

projection

Converts `text` to Unicode (if it's not already), assuming utf-8 input.

r""" Instantiates one of the base model classes of the library from a pre-trained model configuration. The model class to instantiate

Reads a json list file.

Converts a string in a sequence of ids (integer), using the tokenizer and vocabulary. Same as doing ``self.convert_tokens_to_ids(sel

Reads a tab separated value file.

Constructs a `BertConfig` from a json file of parameters.

Prune a linear layer (a model parameters) to keep only entries in index. Return the pruned layer as a new layer with requires_grad=True.

Creates a `truncated_normal_initializer` with the given range.

Tie or clone module weights depending of weither we are using TorchScript or not

Converts a single token, or a sequence of tokens, (str/unicode) in a single integer id (resp. a sequence of ids), using the vocabulary.

Instantiate a PreTrainedBertModel from a pre-trained model file or a pytorch state dict. Download and cache the pre-trained model fil

Project hidden states to a specific head with a 4D-shape.

Save a model and its configuration file to a directory, so that it can be re-loaded using the `:func:`~transformers.PreTrainedModel.from_

Creates a `truncated_normal_initializer` with the given range.

Creates a `truncated_normal_initializer` with the given range.

Creates a `truncated_normal_initializer` with the given range.

Creates a `truncated_normal_initializer` with the given range.

Creates a `truncated_normal_initializer` with the given range.

Creates a `truncated_normal_initializer` with the given range.

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Creates or gets the iterations_per_loop variable. In TPUEstimator, the user provided computation, the model_fn, is wrapped inside a tf.while_loop

Build a resized Embedding Module from a provided token Embedding Module. Increasing the size will add newly initialized vectors at the en

Performs a single optimization step. Arguments: closure (callable, optional): A closure that reevaluates the model

Conv1D layer as defined by Radford et al. for OpenAI GPT (and also used in GPT-2) Basically works like a Linear layer but the weights are

Gets `features` and `labels`.

Basic Tokenization of a piece of text. Split on "white spaces" only, for sub-word tokenization, see WordPieceTokenizer. Args: