A residual block that can optionally change the number of channels. :param channels: the number of input channels. :param emb_channels: t

A sequential module that passes timestep embeddings to the children that support it as an extra input.

An attention block that allows spatial positions to attend to each other. Originally ported from here, but adapted to the N-d case. https

A downsampling layer with an optional convolution. :param channels: channels in the inputs and outputs. :param use_conv: a bool determini

`Large-scale CelebFaces Attributes (CelebA) Dataset <http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html>`_ Dataset. Args: root (string): Ro

`LSUN <https://www.yf.io/p/lsun>`_ dataset. Args: root (string): Root directory for the database files. classes (string or l

Transformer block for image-like data. First, project the input (aka embedding) and reshape to b, t, d. Then apply standard transform

An upsampling layer with an optional convolution. :param channels: channels in the inputs and outputs. :param use_conv: a bool determinin

r""" A basic Transformer block. Parameters: dim (`int`): The number of channels in the input and output. num_attention_heads

r""" A basic Transformer block. Parameters: dim (`int`): The number of channels in the input and output. num_attention_heads

A module which performs QKV attention and splits in a different order.

Adapted from CLIP: https://github.com/openai/CLIP/blob/main/clip/model.py

Uses a pretrained BERT tokenizer by huggingface. Vocab size: 30522 (?)

Cross frame attention processor. Each frame attends the first frame. Args: batch_size: The number that represents actual batch size,

Cross frame attention processor with scaled_dot_product attention of Pytorch 2.0. Args: batch_size: The number that represents actua

Uses the CLIP transformer encoder for text.

Uses the CLIP image encoder.

to match AttnBlock usage

A module which performs QKV attention. Matches legacy QKVAttention + input/ouput heads shaping