The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between

A class that keeps all conversation history.

Fake file-like stream object that redirects writes to a logger instance.

Construct the embeddings from word and position embeddings.

Multi-headed attention from 'Attention Is All You Need' paper

Transformer encoder consisting of `config.num_hidden_layers` self attention layers. Each layer is a [`SigLipEncoderLayer`]. Args:

Multihead Attention Pooling.

Vision Transformer A PyTorch impl of : `An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale` - https://arxiv.org/a

An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models.

Multi-headed attention from 'Attention Is All You Need' paper. Modified to use sliding window attention: Longformer and "Generating Long Sequ

Base class for Qwen2VL causal language model (or autoregressive) outputs. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional

Qwen2VL flash attention module, following Qwen2VL attention module. This module inherits from `Qwen2VLAttention` as the weights of the module

Qwen2 attention module using torch.nn.functional.scaled_dot_product_attention. This module inherits from `Qwen2Attention` as the weights of t

Different separator style.

An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models.

Base class for vision model's outputs that also contains image embeddings of the pooling of the last hidden states. Args: image_embe