Activates gradient checkpointing for the current model. Note that in other frameworks this feature can be referred to as "activation

Return a list mapping the tokens to the id of their original sentences: - `None` for special tokens added around or between sequ

Load flax checkpoints in a PyTorch model

Performs a single optimization step. Arguments: closure (`Callable`, *optional*): A closure that reevaluates the model a

Constructs an identity transformation. Args: shape: The desired shape dtype:

Register a new configuration for this class. Args: model_type (`str`): The model type like "bert" or "gpt".

This class computes the losses using the class predictions, mask predictions and the contrastive queries. Oneformer calculates the c

Args: dimension: fixed_dimension: num_token_to_add: Returns:

Analogous to the to() method of torch Tensors Args: device: A torch device dtype:

Passes over every char (utf-8 char) on word and recursively adds it to the internal `data` trie representation. The special key `""`

Moves the transformation object to GPU memory Returns: A version of the transformation on GPU

Overrides the `from_dict` method from the base class to make sure `reduce_labels` is updated if image processor is created using from

r""" Instantiate a [`EncoderDecoderConfig`] (or a derived class) from a pre-trained encoder model configuration and decoder model conf

Whether or not this process is the global main process (when training in a distributed fashion on several machines, this is only goin

Find the correct padding/truncation strategy with backward compatibility for old arguments (truncation_strategy and pad_to_max_length

r""" Args: batch_size (`int`): batch_size used for fast auto-regressive decoding. Defines the batch size of the in

r""" Generates sequences of token ids for models with a language modeling head using **multinomial sampling** and can be used for text

Infers the channel dimension format of `image`. Args: image (`np.ndarray`): The image to infer the channel dimension of.

Resizes `image`. Enforces conversion of input to PIL.Image. Args: image (`PIL.Image.Image` or `np.ndarray` or `torch.Ten

Converts an index (integer) in a token (str) using the vocab.

Returns the log level to be used depending on whether this process is the main process of node 0, main process of node non-0, or a no

CrossEntropyLoss that ignores pad tokens

Get number of (optionally, trainable or non-embeddings) parameters in the module. Args: only_trainable (`bool`, *optiona

Normalize an image. image = (image - image_mean) / image_std. Args: image (`np.ndarray`): Image to norma

Run pipeline tests for a specific `task` Args: task (`str`): A task name. This should be a key in the mapping `pi

Tokenizes a piece of text into its word pieces. This uses a greedy longest-match-first algorithm to perform tokenization using the gi

Converts a token (str) in an id using the vocab.

Checks whether `chars` is a punctuation character.

Creates a draft of a model card using the information available to the `Trainer`. Args: language (`str`, *optional*):

Checks whether `chars` is a control character.

Checks whether `chars` is a whitespace character.

Crops the `image` to the specified `size` using a center crop. Note that if the image is too small to be cropped to the size given, it will b

Generate an image from the given prompts. If image_path is provided, the image is used as a starting point for the optimization. If im

Pads the `image` with the specified (height, width) `padding` and `mode`. Args: image (`np.ndarray`): The image to pad.

Wraps a HuggingFace [`~datasets.Dataset`] as a `tf.data.Dataset` with collation and batching. This method is designed to create a "re

Args: inputs (`torch.FloatTensor`) The input matrix from which the binarizer computes the binary mask.

require_version wrapper which emits a core-specific hint on failure