Instantiate a PreTrainedBertModel from a pre-trained model file or a pytorch state dict. Download and cache the pre-trained model fil

Performs a single optimization step. Arguments: closure (callable, optional): A closure that reevaluates the model

Converts a sequence of tokens into ids using the vocab.

Runs basic whitespace cleaning and splitting on a peice of text.

Given something that might be a URL (or might be a local path), determine which. If it's a URL, download the file and cache it, and re

r""" Slices tensors into approximately equal chunks and distributes them across given GPUs. Duplicates references to objects that are not

Split a full s3 path into the bucket name and path.

Serializes this instance to a JSON string.

Performs invalid character removal and whitespace cleanup on text.

Checks whether CP is the codepoint of a CJK character.

Checks whether `chars` is a control character.

Checks whether `chars` is a punctuation character.

Checks whether `chars` is a whitespace character.

Load entries img_id2val: dict {img_id -> val} val can be used to retrieve image or features dataroot: root path of dataset name: 'train',

Load entries img_id2val: dict {img_id -> val} val can be used to retrieve image or features dataroot: root path of dataset name: 'train',

Splits punctuation on a piece of text.

Strips accents from a piece of text.

Adds whitespace around any CJK character.

Converts a sequence of ids in wordpiece tokens using the vocab.

Constructs a `BertConfig` from a Python dictionary of parameters.

Constructs a `BertConfig` from a json file of parameters.

Given a URL, look for the corresponding dataset in the local cache. If it's not there, download it. Then return the path to the cached file