A residual block that can optionally change the number of channels. :param channels: the number of input channels. :param emb_channels: t

A sequential module that passes timestep embeddings to the children that support it as an extra input.

An attention block that allows spatial positions to attend to each other. Originally ported from here, but adapted to the N-d case. https

A downsampling layer with an optional convolution. :param channels: channels in the inputs and outputs. :param use_conv: a bool determini

Transformer block for image-like data. First, project the input (aka embedding) and reshape to b, t, d. Then apply standard transform

An upsampling layer with an optional convolution. :param channels: channels in the inputs and outputs. :param use_conv: a bool determinin

1 Dimensional version of data (fmri voxels) to Patch Embedding

A module which performs QKV attention and splits in a different order.

Adapted from CLIP: https://github.com/openai/CLIP/blob/main/clip/model.py

Uses a pretrained BERT tokenizer by huggingface. Vocab size: 30522 (?)

Uses the CLIP transformer encoder for text (from Hugging Face)

to match AttnBlock usage

Masked Autoencoder with VisionTransformer backbone

A module which performs QKV attention. Matches legacy QKVAttention + input/ouput heads shaping

Improved version over VectorQuantizer in taming, can be used as a drop-in replacement. Mostly avoids costly matrix multiplications and allows

Uses the BERT tokenizr model and add some transformer encoder layers

main class

The half UNet model with attention and timestep embedding. For usage, see UNet.

Uses the CLIP transformer encoder for text (from Hugging Face)

Uses the CLIP transformer encoder for text.

Uses the CLIP image encoder.

main class

Any module where forward() takes timestep embeddings as a second argument.

Some transformer encoder layers

Learned 2x upsampling without padding

The full UNet model with attention and timestep embedding. :param in_channels: channels in the input Tensor. :param model_channels: base