Causal 3d convolusion.

r""" A custom 3D causal convolution layer with feature caching support. This layer extends the standard Conv3D layer by ensuring causality in

Causal self-attention with a single head.

r""" Processor for implementing scaled dot-product attention (enabled by default if you're using PyTorch 2.0).

r""" A custom RMS normalization layer. Args: dim (int): The number of dimensions to normalize over. channel_first (bool, opti

An attention layer that allows for downscaling the size of the embedding after projection to queries, keys, and values.

r""" A custom residual block module. Args: in_dim (int): Number of input channels. out_dim (int): Number of output channels.

r""" A Resnet block. Parameters: in_channels (`int`): The number of channels in the input. out_channels (`int`, *optional*, d

A structure for storing masks and their related data in batched format. Implements basic filtering and concatenation.

A 2D downsampling layer with an optional convolution. Parameters: channels (`int`): number of channels in the inputs and outp

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Flux 2 uses a SwiGLU-style activation in the transformer feedforward sub-blocks, but with the linear projection layer fused into the first li

r""" Causal self-attention with a single head. Args: dim (int): The number of channels in the input tensor.

Middle block for WanVAE encoder and decoder. Args: dim (int): Number of input/output channels. dropout (float): Dropout rate

r""" A custom resampling module for 2D and 3D data. Args: dim (int): The number of input/output channels. mode (str): The res

r""" Perform upsampling while ensuring the output tensor has the same data type as the input. Args: x (torch.Tensor): Input tensor to

Residual convolution module.

Resize sample to given size (width, height).

A 2D UNet mid-block [`UNetMidBlock2D`] with multiple residual blocks and optional attention blocks. Args: in_channels (`int`): The n

A 2D upsampling layer with an optional convolution. Parameters: channels (`int`): number of channels in the inputs and output

XLMRobertaModel with no pooler and no LM head.