Instantiate a PreTrainedBertModel from a pre-trained model file. Download and cache the pre-trained model file if needed.

Converts a sequence of tokens into ids using the vocab.

Runs basic whitespace cleaning and splitting on a piece of text.

Given something that might be a URL (or might be a local path), determine which. If it's a URL, download the file and cache it, and retur

Constructs a `BertConfig` from a json file of parameters.

Load tf checkpoints in a pytorch model

Split a full s3 path into the bucket name and path.

Serializes this instance to a JSON string.

Performs invalid character removal and whitespace cleanup on text.

Checks whether CP is the codepoint of a CJK character.

Checks whether `chars` is a control character.

Checks whether `chars` is a punctuation character.

Checks whether `chars` is a whitespace character.

Splits punctuation on a piece of text.

Strips accents from a piece of text.

Adds whitespace around any CJK character.

Loads a data file into a list of `InputFeature`s.

Creates the predictions for the masked LM objective. This is mostly copied from the Google BERT repo, but with several refactors to clean it up an

Constructs a `BertConfig` from a Python dictionary of parameters.

Given a URL, look for the corresponding dataset in the local cache. If it's not there, download it. Then return the path to the cached file.

Loads a vocabulary file into a dictionary.

Read a list of `InputExample`s from an input file.

Check ETag on S3 object.

Pull a file directly from S3.

Compute softmax values for each sets of scores in x.

Compute softmax values for each sets of scores in x.

Performs a single optimization step. Arguments: closure (callable, optional): A closure that reevaluates the model

Serializes this instance to a Python dictionary.

Truncates a pair of sequences to a maximum sequence length. Lifted from Google's BERT repo.

Convert `url` into a hashed filename in a repeatable way. If `etag` is specified, append its hash to the url's, delimited by a period.

Specifies a triangular learning rate schedule where peak is reached at `warmup`*`t_total`-th (as provided to BertAdam) training step. After `

Constructs a BertTokenizer. Args: vocab_file: Path to a one-wordpiece-per-line vocabulary file do_lower_case: Whether to

Constructs a BasicTokenizer. Args: do_lower_case: Whether to lower case the input.

Constructs BertConfig. Args: vocab_size_or_config_json_file: Vocabulary size of `inputs_ids` in `BertModel`. hidden_s

Construct a layernorm module in the TF style (epsilon inside the square root).