Tests if `x` is a Jax tensor or not. Safe to call even if jax is not installed.

Iterates over all blobs of the conversation. Returns: Iterator of (is_user, text_chunk) in chronological order of the conversation.

Rescores a prediction based on penalty functions

Framework-agnostic version of `numpy.transpose` that will work on torch/TensorFlow/Jax tensors as well as NumPy arrays.

Computes random mask spans for a given shape. Used to implement [SpecAugment: A Simple Data Augmentation Method for ASR](https://arxiv.org/ab

splits sequence length dim of vectors into `dim_factor_1` and `dim_factor_2` dims

Add `arrays` to the internal storage, Will initialize the storage to the full size at the first arrays passed so that if we're bound

Convert the inner content to tensors. Args: tensor_type (`str` or [`~utils.TensorType`], *optional*): Th

r""" Returns: Example: ```python >>> from transformers import AutoTokenizer, FlaxBartForConditionalGeneration

Creates a random float32 tensor

Returns a dictionary of layers distributed evenly across all devices.

Create a schedule with a learning rate that decreases linearly from the initial lr set in the optimizer to 0, after a warmup period during wh

CrossEntropyLoss that ignores pad tokens

Concat the `new_tensors` to `tensors` on the first dim and pad them on the second if needed. Works for tensors or nested list/tuples/dict of

Set the random seed.

projection

Converts `image` to a PIL Image. Optionally rescales it and puts the channel dimension back as the last axis if needed. Args

If tensors not close, or a and b arent both tensors, raise a nice Assertion error.

Calculate rouge using rouge_scorer package. Args: pred_lns: list of summaries generated by model tgt_lns: list of groundtruth sum

Returns a copy of a function f.

Creates an optimizer with a learning rate schedule using a warmup phase followed by a linear decay. Args: init_lr (`float`):

This method forwards all its arguments to PreTrainedTokenizer's [`~PreTrainedTokenizer.decode`]. Please refer to the docstring of thi

`tokenizers.decoders.Decoder`: The Rust decoder for this tokenizer.

r""" Instantiate a [`VisionTextDualEncoderConfig`] (or a derived class) from text model configuration and vision model configuration.

Normalizes `image` with `mean` and `std`. Note that this will trigger a conversion of `image` to a NumPy array if it's a PIL Image.

A method executed at the end of each Transformer model initialization, to execute code that needs the model's modules properly initia

Prepares a sequence of input id, entity id and entity span, or a pair of sequences of inputs ids, entity ids, entity spans so that it