Layer scaling module for stable training.

DINOv3 Vision Transformer for visual representation learning.

DINOv3-based Pixel Decoder for image reconstruction. This decoder uses a transformer architecture with RoPE position embeddings and PixelShuf

A single linear layer which does a 1x1 conv

Unified DINO Head supporting both weight_norm and flexible forward modes. This head is used for self-supervised learning in DINO/iBOT style train

DINOv3 Vision Transformer with feature bottleneck for dimensionality reduction. This variant adds a linear projection layer to reduce the feature

Root Mean Square Layer Normalization.

Residual attention block for CLIP-style transformers.

RoPE positional embedding with no mixing of coordinates (axial) and no learnable weights. Supports two parametrizations of the rope parameters: e

Self-attention block with FFN for DINOv3.

Text Transformer for CLIP-style text encoding.

Container for multiple linear classifiers.

Multi-head attention module for CLIP-style transformers.

Causal self-attention for autoregressive models.

Custom residual attention block with scaled cosine attention.

Wrapper that extracts intermediate layer features from VTPModel.

Wraps another sampler to yield an infinite stream of indices.

LPIPS metric wrapper.

Linear classifier on top of frozen features.

Standard MLP layer with optional dropout.

2D image to patch embedding: (B,C,H,W) -> (B,N,D) Args: img_size: Image size. patch_size: Patch token size. in_chans

Simple tokenizer for CLIP models.

Base Transformer for text encoding.

Attentional pooling for multi-head attention.

Causal self-attention block for autoregressive models.

Cosine annealing scheduler with optional warmup and freeze periods. Supports linear warmup followed by cosine annealing decay with optional

A custom transformer that can use different block types.

Learned perceptual metric.

Subclass torch's LayerNorm (with cast back to input dtype).

Subclass torch's LayerNorm to handle fp16 (by casting to float32 and back).

Linear layer with masked bias for K in QKV attention.

Mixin for forward_list operation on list of tensors.

Multimodal Transformer with cross-attention for vision-language tasks.

Patch dropout for vision transformers. Reference: https://arxiv.org/abs/2212.00794

Quick GELU activation function. NOTE: This is slower than nn.GELU or nn.SiLU and uses more GPU memory.

Self-attention module with optional RoPE support.

SwiGLU Feed-Forward Network. Reference: GLU Variants Improve Transformer (https://arxiv.org/abs/2002.05202)

Visual Tokenizer Pre-training Model. A unified framework for vision-language pre-training that supports: - Contrastive learning (CLIP-style)

Configuration class for VTP (Visual Tokenizer Pre-training) model. This configuration supports two main modes: - CLIP-style contrastive learn

VTP (Visual Tokenizer Pre-training) Model. A unified framework supporting multiple vision tasks through composable methods: Basic feature ex

Base class for VTP models.