A residual block that can optionally change the number of channels. :param channels: the number of input channels. :param emb_channels: t

A sequential module that passes timestep embeddings to the children that support it as an extra input.

An upsampling layer with an optional convolution. :param channels: channels in the inputs and outputs. :param use_conv: a bool determinin

A downsampling layer with an optional convolution. :param channels: channels in the inputs and outputs. :param use_conv: a bool determini

Transformer block for image-like data in temporal axis. First, reshape to b, t, d. Then apply standard transformer action. Finally, r

Transformer block for image-like data in spatial axis. First, project the input (aka embedding) and reshape to b, t, d. Then apply st

https://github.com/evelinehong/Transformer_Relative_Position_PyTorch/blob/master/relative_position.py

Uses the CLIP transformer encoder for text (from huggingface)

Uses the T5 transformer encoder for text

to match AttnBlock usage

to match AttnBlock usage

Uses xformers efficient implementation, see https://github.com/MatthieuTPHR/diffusers/blob/d80b531ff8060ec1ea982b65a1b8df70f73aa67c/src/dif

A residual block that can optionally change the number of channels. :param channels: the number of input channels. :param emb_channels

Adapted from modelscope: https://github.com/modelscope/modelscope/blob/master/modelscope/models/multi_modal/video_synthesis/unet_sd.py

WebVid Dataset. Assumes webvid data is structured as follows. Webvid/ videos/ 000001_000050/ ($page_dir)

Wraps an arbitrary object with __len__ and __getitem__ into a pytorch dataset

Uses the OpenCLIP transformer encoder for text

Uses the OpenCLIP vision transformer encoder for images

Uses the OpenCLIP vision transformer encoder for images

Projection Model

main class