Given something that might be a URL (or might be a local path), determine which. If it's a URL, download the file and cache it, and retur

Instantiate a PreTrainedBertModel from a pre-trained model file. Download and cache the pre-trained model file if needed.

获取已使用时间

Update input embeddings with new embedding matrice if needed

Serializes this instance to a JSON string.

Serializes this instance to a JSON string.

Serializes this instance to a JSON string.

Runs basic whitespace cleaning and splitting on a piece of text.

Constructs a `BertConfig` from a json file of parameters.

Return set of symbol pairs in a word. word is represented as tuple of symbols (symbols being variable-length strings)

Return set of symbol pairs in a word. Word is represented as tuple of symbols (symbols being variable-length strings).

Load tf checkpoints in a pytorch model

Load tf checkpoints in a pytorch model

Load tf pre-trained weights in a pytorch model (from NumPy arrays here)

Load tf checkpoints in a pytorch model

embedding: an nn.Embedding layer bias: [n_vocab] labels: [b1, b2] inputs: [b1, b2, n_emb] sampler: you may us

Split a full s3 path into the bucket name and path.

Run this to be sure output and input (adaptive) softmax weights are tied

Serializes this instance to a JSON string.

Reference : https://github.com/tensorflow/tensorflow/blob/r1.10/tensorflow/python/ops/candidate_sampling_ops.py `P(class) = (log(

Performs invalid character removal and whitespace cleanup on text.

Checks whether CP is the codepoint of a CJK character.

Checks whether `chars` is a control character.

Checks whether `chars` is a punctuation character.

Checks whether `chars` is a whitespace character.

Splits punctuation on a piece of text.

Strips accents from a piece of text.

Adds whitespace around any CJK character.

A map of modules from TF to PyTorch. This time I use a map to keep the PyTorch model as identical to the original PyTorch model as possible.

Returns list of utf-8 byte and a corresponding list of unicode strings. The reversible bpe codes work on unicode strings. This means you

Converts a sequence of ids in BPE tokens using the vocab.

Converts a sequence of tokens into ids using the vocab.

Converts a sequence of tokens into ids using the vocab.

Converts a sequence of tokens into ids using the vocab.

Converts a sequence of symbols into ids using the vocab.

Constructs a `TransfoXLConfig` from a Python dictionary of parameters.

Constructs a `GPT2Config` from a Python dictionary of parameters.

Constructs a `BertConfig` from a Python dictionary of parameters.

Constructs a `OpenAIGPTConfig` from a Python dictionary of parameters.

Constructs a `TransfoXLConfig` from a json file of parameters.

Constructs a `GPT2Config` from a json file of parameters.

Constructs a `OpenAIGPTConfig` from a json file of parameters.

Given a URL, look for the corresponding dataset in the local cache. If it's not there, download it. Then return the path to the cached file.

:param progress: value between 0 and 1 (unless going beyond t_total steps) specifying training progress :return: learni

Initialize the weights.