Constructs a BertTokenizer. Args: **vocab_file**: Path to a one-wordpiece-per-line vocabulary file **do_lower_case**:

Constructs a BasicTokenizer. Args: **do_lower_case**: Whether to lower case the input. **never_split**: (`optional`)

Constructs GPT2Config. Args: vocab_size_or_config_json_file: Vocabulary size of `inputs_ids` in `GPT2Model` or a configuration js

Size of the full vocabulary with the added tokens

Converts an index (integer) in a token (string/unicode) using the vocab.

Converts an index (integer) in a token (string/unicode) using the vocab.

Converts a token (str/unicode) in an id using the vocab.

Build a resized Embedding Module from a provided token Embedding Module. Increasing the size will add newly initialized vectors at the en

Initialize the weights

Initialize the weights

Initialize the weights.

Initialize the weights.

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTraine

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTraine

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer}

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer}

Tokenize a string.

A new main code is reached and its name is added.

Adds special tokens to a sequence pair for sequence classification tasks. A BERT sequence pair has the following format: [CLS] A [SEP

Adds special tokens to the a sequence for sequence classification tasks. A BERT sequence has the following format: [CLS] X [SEP]

All the additional special tokens you may want to use (list of strings). Log an error if used while not having been set.

Ids of all the additional special tokens in the vocabulary (list of integers). Log an error if used while not having been set.

List the vocabulary indices of the special tokens ('<unk>', '<cls>'...) mapped to class attributes (cls_token, unk_token...).

List all the special tokens ('<unk>', '<cls>'...) mapped to class attributes (cls_token, unk_token...).

Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [

Beginning of sentence token (string). Log an error if used while not having been set.

Id of the beginning of sentence token in the vocabulary. Log an error if used while not having been set.

compute the number of active units

Returns: Tensor1 Tensor1: the mean of inference distribution, with shape [batch, nz]

compute the mean value of model posterior, i.e. E_{z ~ p(z|x)}[z] Args: grid_z: different z points that will be evaluated, with

Classification token (string). E.g. to extract a summary of an input sequence leveraging self-attention along the full depth of the model. Log an err

Id of the classification token in the vocabulary. E.g. to extract a summary of an input sequence leveraging self-attention along the full depth of th

Convert the torso of the model to float16.

Convert the torso of the model to float16.

Convert the torso of the model to float32.