Data class for intermediate outputs of BLIP models. image_embeds (torch.FloatTensor): Image embeddings, shape (batch_size, num_patches, embe

Subclass torch's LayerNorm to handle fp16.

Track a series of values and provide access to smoothed values over a window or the global series average.

Subclass torch's LayerNorm to handle fp16.

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between

Data class of features from BlipFeatureExtractor. Args: image_embeds: (torch.FloatTensor) of shape (batch_size, num_patches+1, embed

Data class of features from AlbefFeatureExtractor. Args: image_embeds: `torch.FloatTensor` of shape `(batch_size, 1, embed_dim)`, `o

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig

MLP as used in Vision Transformer, MLP-Mixer and related networks

Image to Patch Embedding

Args: clip (torch.tensor, dtype=torch.uint8): Size is (C, T, H, W) Return: clip (torch.tensor, dtype=torch.float): Size is (T

Args: clip (torch.tensor, dtype=torch.uint8): Size is (C, T, H, W) Return: clip (torch.tensor, dtype=torch.float): Size is (T

Data class of features from AlbefFeatureExtractor. Args: image_embeds: `torch.FloatTensor` of shape `(batch_size, num_patches+1, emb

Attention based 2D feature pooling w/ learned (absolute) pos embedding. This is a multi-head attention based replacement for (spatial) average poo

Construct the embeddings from word and position embeddings.

Construct the embeddings from word and position embeddings.

Construct the embeddings from word and position embeddings.

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between

Class for BLIP feature extractor. Supported model types: - base: BLIP base model with pre-trained weights from capfilt by BLIP large

This is a preliminary implementation to centralize and validate the configuration. May be altered in the future. A helper class to valid

A wrapper to convert DataLoader as an infinite iterator. Modified from: https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/i

Subclass torch's LayerNorm to handle fp16.

Multi-headed attention from 'Attention Is All You Need' paper

A utility class to create string representation of available model architectures and types. >>> from lavis.models import model_zoo >>> #

A ResNet class that is similar to torchvision's but contains the following changes: - There are now 3 "stem" convolutions as opposed to 1, wi