BERT model ("Bidirectional Embedding Representations from a Transformer"). Params: config: a BertConfig class instance with the configura

A quite easy output class for bert output, in order to enable this Bert modle to be used like the Bert model in transformers

Runs basic tokenization (punctuation splitting, lower casing, etc.).

Configuration class to store the configuration of a `BertModel`.

Construct the embeddings from word, position and token_type embeddings.

BERT model with pre-training heads. This module comprises the BERT model followed by the two pre-training heads: - the masked language mod

convert pyarrow.Table to python dict

Runs WordPiece tokenization.

Implements BERT version of Adam algorithm with weight decay fix. Params: lr: learning rate warmup: portion of t_total for the warm

BERT model with the masked language modeling head. This module comprises the BERT model followed by the masked language modeling head. Params

BERT model for multiple choice tasks. This module is composed of the BERT model with a linear layer on top of the pooled output. Params:

BERT model with next sentence prediction head. This module comprises the BERT model followed by the next sentence classification head. Params

BERT model for Question Answering (span extraction). This module is composed of the BERT model with a linear layer on top of the sequence outp

BERT model for classification. This module is composed of the BERT model with a linear layer on top of the pooled output. Params:

BERT model for token-level classification. This module is composed of the BERT model with a linear layer on top of the full hidden state of th

An abstract class to handle weights initialization and a simple interface for dowloading and loading pretrained models.

Runs end-to-end tokenization: punctuation splitting + wordpiece