A sequential module that passes timestep embeddings to the children that support it as an extra input.

A residual block that can optionally change the number of channels. :param channels: the number of input channels. :param emb_channels

Normlize image by given mean and std.

An attention block that allows spatial positions to attend to each other. Originally ported from here, but adapted to the N-d case. ht

Feature fusion block.

Transformer block for image-like data. First, project the input (aka embedding) and reshape to b, t, d. Then apply standard trans

A downsampling layer with an optional convolution. :param channels: channels in the inputs and outputs. :param use_conv: a bool determ

Feature fusion block.

Interpolation module.

An upsampling layer with an optional convolution. :param channels: channels in the inputs and outputs. :param use_conv: a bool determi

Prepare sample for usage as network input.

A module which performs QKV attention and splits in a different order.

Residual convolution module.

Residual convolution module.

Resize sample to given size (width, height).

Uses the CLIP transformer encoder for text (from huggingface)

Uses the T5 transformer encoder for text

Uses xformers efficient implementation, see https://github.com/MatthieuTPHR/diffusers/blob/d80b531ff8060ec1ea982b65a1b8df70f73aa67c

Network for monocular depth estimation.

Network for monocular depth estimation.

A module which performs QKV attention. Matches legacy QKVAttention + input/ouput heads shaping

Adapted from CLIP: https://github.com/openai/CLIP/blob/main/clip/model.py

Uses the OpenCLIP transformer encoder for text

condition on monocular depth estimation

main class

Basis for different finetunas, such as inpainting or depth2image To disable finetuning mode, set finetune_keys to None

can either run as pure inpainting model (only concat mode) or with mixed conditionings, e.g. mask as concat and text via cross-attn. T

condition on low-res image (and optionally on some spatial noise augmentation)

Any module where forward() takes timestep embeddings as a second argument.

Learned 2x upsampling without padding

The full UNet model with attention and timestep embedding. :param in_channels: channels in the input Tensor. :param model_channels: ba