AddedToken represents a token to be added to a Tokenizer An AddedToken can have special options defining the way it should behave.

Module class that surfaces all objects but only performs associated imports when the objects are requested.

r""" Holds the output of the image processor specific `__call__` methods. This class is derived from a python dictionary and can be used as a

Base class for model's outputs, with potential hidden states and attentions. Args: last_hidden_state (`torch.FloatTensor` of shape `

This is a wrapper class about all possible attributes and features that you can play with a model that has been loaded using `bitsandbytes`.

Trainer is a simple but feature-complete training and eval loop for PyTorch, optimized for 🤗 Transformers. Args: model ([`PreTrained

TrainingArguments is the subset of the arguments we use in our example scripts **which relate to the training loop itself**. Using [`HfA

Class that holds a configuration for a generation task. A `generate` call supports the following generation methods for text-decoder, text-to

Base class for model's outputs that may also contain a past key/values (to speed up sequential decoding). Args: last_hidden_state (`

Configuration for recording outputs from a model via hooks. Attributes: target_class (Type): The class (e.g., nn.Module) to which th

Base class for causal language model (or autoregressive) outputs. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, retur

Base class for model's outputs that may also contain a past key/values (to speed up sequential decoding). Args: last_hidden_state (`

Base class for outputs of sentence classification models. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when

Holds the output of the [`~tokenization_utils_base.PreTrainedTokenizerBase.__call__`], [`~tokenization_utils_base.PreTrainedTokenizerBase.enc

Base class for outputs of token classification models. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `l

Base class for model's outputs that also contains a pooling of the last hidden states. Args: last_hidden_state (`torch.FloatTensor`

Context manager to capture `logging` streams Args: logger: 'logging` logger object Returns: The captured output is avai

Base class for outputs of question answering models. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `lab

Hashable dictionary to store image size information.

This subclass of `argparse.ArgumentParser` uses type hints on dataclasses to generate arguments. The class is designed to play well with the

" A mapping config to object (model or tokenizer for instance) that will load keys and values when it is accessed. Args: - config_ma

Base class for masked language models outputs. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` i

r""" This is the configuration class to store the configuration of a [`LlamaModel`]. It is used to instantiate an LLaMA model according to the

Base class for causal language model (or autoregressive) outputs. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, retur

Data collator used for language modeling. Inputs are dynamically padded to the maximum length of a batch if they are not all of the same leng

Create a temporary Hub repository and return its `RepoUrl` object. This is similar to `tempfile.TemporaryDirectory` and can be used as a context m

Base class for outputs of multiple choice models. Args: loss (`torch.FloatTensor` of shape *(1,)*, *optional*, returned when `labels

Base class for sequence-to-sequence language models outputs. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned w

r""" Construct a BERT tokenizer. Based on WordPiece. This tokenizer inherits from [`PreTrainedTokenizer`] which contains most of the main met

r""" Base class for all configuration classes. Handles a few parameters common to all models' configurations as well as methods for loading/do

Base class for model's outputs, with potential hidden states and attentions. Args: last_hidden_state (`torch.FloatTensor` of shape `

Base class for model's outputs that also contains a pooling of the last hidden states. Args: last_hidden_state (`torch.FloatTensor`

Constructs a BasicTokenizer that will run basic tokenization (punctuation splitting, lower casing, etc.). Args: do_lower_case (`bool

Data collator that will dynamically pad the inputs received, as well as the labels. Args: tokenizer ([`PreTrainedTokenizer`] or [`Pr

A Linear layer with built-in nonstandard initializations. Called just like torch.nn.Linear. Implements the initializers in 1.11.4, plus some

Dataclass that holds extra useful data for processing multimodal data. Processors currently cannot return keys, unless it is used in mode

A 3D rotation. Depending on how the object is initialized, the rotation is represented by either a rotation matrix or a quaternion, though bo

Base class for model encoder's outputs that also contains : pre-computed hidden states that can speed up sequential decoding. Args:

r""" This is the configuration class to store the configuration of a [`BertModel`] or a [`TFBertModel`]. It is used to instantiate a BERT mode

This is a wrapper class about all possible attributes and features that you can play with a model that has been loaded using mxfp4 quantizati

Static Cache class to be used with `torch.compile(model)` and `torch.export()`. It will check the `config` for potential hybrid cache structu

Base class for outputs of backbones. Args: feature_maps (`tuple(torch.FloatTensor)` of shape `(batch_size, num_channels, height, wid

Data collator that will dynamically pad the inputs received. Args: tokenizer ([`PreTrainedTokenizer`] or [`PreTrainedTokenizerFast`]

Base class for outputs of image classification models. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `l

Base class for causal language model (or autoregressive) with mixture of experts outputs. Args: loss (`torch.FloatTensor` of shape `

Base class for model's outputs, with potential hidden states. Args: last_hidden_state (`torch.FloatTensor` of shape `(batch_size, nu

Data collator that will dynamically pad the inputs received, as well as the labels. Args: tokenizer ([`PreTrainedTokenizer`] or [`Pr

A single training/test example for simple sequence classification. Args: guid: Unique id for the example. text_a: string. Th

Base class for outputs of sentence classification models. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when

A cache that grows dynamically as more tokens are generated. This is the default for generative models. It stores the key and value states as

This is a wrapper class about all possible attributes and features that you can play with a model that has been loaded using `auto-awq` libra

r""" [`VisionEncoderDecoderModel`] is a generic model class that will be instantiated as a transformer architecture with one of the base visio

Data collator used for padding free approach. Does the following: - concatenates the entire mini batch into single long sequence of shape [1

Base class for outputs of image classification models. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `l

Internally used error class for signalling an optional dependency was not found.

Quantizes the given activation. Args: activation_bit (`int`): Bitwidth for the quantized activation. act_range_m

A class representing a rigid transformation. Little more than a wrapper around two objects: a Rotation object and a [*, 3] translation Design

This is a wrapper class about all possible attributes and features that you can play with a model that has been loaded using `optimum` api fo

r""" This is the configuration class to store the configuration of a [`PatchTSMixerModel`]. It is used to instantiate a PatchTSMixer model acc

Base class for causal language model (or autoregressive) outputs. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, retur

FP4 quantization using fbgemm kernels

A recipe module designed to make a `PreTrainedModel` exportable with `torch.export`, specifically for decoder-only LM with cache. This module

Base class for model's outputs that also contains a pooling of the last hidden states. Args: last_hidden_state (`torch.FloatTensor`

r""" [`EncoderDecoderModel`] is a generic model class that will be instantiated as a transformer architecture with one of the base model class

This is the configuration class to store the configuration of a [`GPT2Model`] or a [`TFGPT2Model`]. It is used to instantiate a GPT-2 model a

Mixin that contain utilities for preparing image features.

Data collator used for language modeling that masks entire words. - collates batches of tensors, honoring their tokenizer's pad_token -

Index grouping entries within a tensor.

r""" Constructs a Wav2Vec2 processor which wraps a Wav2Vec2 feature extractor and a Wav2Vec2 CTC tokenizer into a single processor. [`Wav

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added betwee

r""" Constructs a LayoutLMv3 image processor. Args: do_resize (`bool`, *optional*, defaults to `True`): Whether to resize