Create a schedule with a learning rate that decreases linearly from the initial lr set in the optimizer to 0, after a warmup period during wh

The real training loop.

Add a dictionary of special tokens (eos, pad, cls, etc.) to the encoder and link them to class attributes. If special tokens are NOT

Get the indentation of `line`.

Whether or not the current process is the local process, based on `xm.get_ordinal()` (for TPUs) first, then on `local_rank`.

Whether or not this process is the global main process (when training in a distributed fashion on several machines, this is only goin

Checks whether `char` is a punctuation character.

Constructs a `ModelCard` from a Python dictionary of parameters.

Deal with dynamic shape in tensorflow cleanly.

r""" Instantiates one of the base model classes of the library from a configuration. Note: Loading a model from its confi

Makes broadcastable attention and causal masks so that future and masked tokens are ignored. Arguments: attention_mask (

Just EOS

Checks whether `char` is a control character.

splits sequence length dim of vectors into `dim_factor_1` and `dim_factor_2` dims

Calculate rouge using rouge_scorer package. Args: pred_lns: list of summaries generated by model tgt_lns: list of groundtruth sum

Finds the heads and their indices taking :obj:`already_pruned_heads` into account. Args: heads (:obj:`List[int]`): List of the indic

This function chunks the :obj:`input_tensors` into smaller input tensor parts of size :obj:`chunk_size` over the dimension :obj:`chunk_dim`.

Function arguments can be inserted as boolean tensor and bool variables to cope with Keras serialization we need to cast the bool arguments (

Creates a random float32 tensor

Cover moses empty string edge case. They return empty list for '' input!

Shift input ids one token to the right, and wrap the last non pad token (usually <eos>).

Alternative helper method that does not use `argparse` at all, instead loading a json file and populating the dataclass types.

Serializes this instance to a JSON string.

The main method to update the progress bar to :obj:`value`. Args: value (:obj:`int`): The value to use.

Set requires_grad=False for each of model.parameters()

list(map(f, x))

Serializes this instance to a Python dictionary.

Tokenizes a piece of text into its word pieces. This uses a greedy longest-match-first algorithm to perform tokenization using the gi

Get the encoded token span corresponding to a word in the sequence of the batch. Token spans are returned as a :class:`~transformers

convert into overlapping chunks. Chunk size = 2w, overlap size = w

See base class.

Retrieves documents for specified ``question_hidden_states``. Args: question_hidden_states (:obj:`np.ndarray` of shape :