Resets the accumulated gradients on the current replica.

Runs `command` with `subprocess.check_output` and will potentially return the `stdout`. Will also properly capture if an error occured while

Shift input ids one token to the right, and wrap the last non pad token (the <LID> token) Note that MBart does not have a single `decoder_sta

Serializes this instance to a Python dictionary. Override the default *to_dict()* from *PretrainedConfig*. Returns: `Dic

Serializes this instance to a Python dictionary. Override the default *to_dict()* from *PretrainedConfig*. Returns: `Dic

convert into overlapping chunks. Chunk size = 2w, overlap size = w

Splits hidden dim into attn_head_size and num_attention_heads

Add a callback to the current list of [`~transformer.TrainerCallback`]. Args: callback (`type` or [`~transformer.TrainerC

Small helper to check pseudo-equality of parsed arguments on `ArgumentParser` instances.

Returns whether this model can generate sequences with `.generate()`. Returns: `bool`: Whether this model can generate s

Save the modeling files corresponding to a custom model/configuration/tokenizer etc. in a given folder. Optionally adds the proper fields in

Returns the device on which the Rigid's tensors are located. Returns: The device on which the Rigid's tensors are locate

Return the properly gathered arrays and truncate to the number of samples (since the sampler added some extras to get each process a

Creates a random float32 tensor

Creates a random float32 tensor

Set requires_grad=False for each of model.parameters()

Overrides the `from_dict` method from the base class to make sure parameters are updated if image processor is created using from_dic

r""" Instantiates an question encoder and a generator from one or two base classes of the library from pretrained model checkpoints.

Returns the model's input embeddings layer. Returns: `tf.Variable`: The embeddings layer mapping vocabulary to hidden st

The LM Head layer. This method must be overwritten by all the models that have a lm head. Return: `tf.keras.layers.Layer

Checks if a repo contains a given file without downloading it. Works for remote repos and local folders. <Tip warning={false}> This fun

Checks if the dataset implements __len__() and it doesn't raise an error

Numpify `tensors` (even if it's a nested list/tuple/dict of tensors).

When used in normal mode, this method forwards all its arguments to MCTCTFeatureExtractor's [`~MCTCTFeatureExtractor.pad`] and return

Prune a Conv1D layer to keep only entries in index. A Conv1D work as a Linear layer (see e.g. BERT) but the weights are transposed. Used

Retrieves documents for specified `question_hidden_states`. Args: question_hidden_states (`np.ndarray` of shape `(batch_

Perform pooling by stride slicing the tensor along the given axis.

Perform pooling by stride slicing the tensor along the given axis.

Serializes this instance to a Python dictionary. Returns: `Dict[str, Any]`: Dictionary of all the attributes that make u