Restores part of the model from other directory. Useful to initialize part of the model with another pretrained model. Args:

Marks that all of the items have been preprocessed. Save state to disk. Used in preprocess.py, after reading all of the data.

Save a model and its configuration file to a directory, so that it can be re-loaded using the `:func:`~transformers.PreTrainedModel.from_pret

Save a model and its configuration file to a directory, so that it can be re-loaded using the `:func:`~transformers.PreTrainedModel.from_pret

Save a model and its configuration file to a directory, so that it can be re-loaded using the `:func:`~transformers.PreTrainedModel.from_pret

Save a model and its configuration file to a directory, so that it can be re-loaded using the `:func:`~transformers.PreTrainedModel.from_pret

Save a model and its configuration file to a directory, so that it can be re-loaded using the `:func:`~transformers.PreTrainedModel.from_pret

Compute sigmoid values

Compute softmax values for each sets of scores in x.

Compute 'Scaled Dot Product Attention

Compute 'Scaled Dot Product Attention

Transfer the pretrained embeddings to the given device.

Given a string, return a list of tokens suitable for lookup.

Decorator to make all processes in distributed training wait for each local_master to do something.

Decorator to make all processes in distributed training wait for each local_master to do something.

Truncates a sequence pair in place to the maximum length.

Undo tokenize.

Checks whether item can be successfully preprocessed. Returns a boolean and an arbitrary object.

Args: matrix (): attention ():