The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between

r""" Swin Transformer A PyTorch impl of : `Swin Transformer: Hierarchical Vision Transformer using Shifted Windows` - https://arxiv

r""" Image to Patch Embedding Args: img_size (int): Image size. Default: 224. patch_size (int): Patch token size. Default: 4.

Vision Transformer A PyTorch impl of : `An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale` - https://arxiv.org/

A basic Swin Transformer layer for one stage. Args: dim (int): Number of input channels. input_resolution (tuple[int]): Input re

A basic Swin Transformer layer for one stage. Args: dim (int): Number of input channels. input_resolution (tuple[int]): Input re

Construct the embeddings from word and position embeddings.

Construct the embeddings from word and position embeddings.

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between

MLP as used in Vision Transformer, MLP-Mixer and related networks

r""" Image to Patch Embedding Args: img_size (int): Image size. Default: 224. patch_size (int): Patch token size. Default: 4.

r""" Swin Transformer Block. Args: dim (int): Number of input channels. input_resolution (tuple[int]): Input resulotion.

r""" Swin Transformer Block. Args: dim (int): Number of input channels. input_resolution (tuple[int]): Input resulotion.

The output of [`UNet2DConditionModel`]. Args: sample (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`):

r""" Window based multi-head self attention (W-MSA) module with relative position bias. It supports both of shifted and non-shifted window. A

r""" Window based multi-head self attention (W-MSA) module with relative position bias. It supports both of shifted and non-shifted window. A

r""" A VAE model with KL loss for encoding images into latents and decoding latent representations into images. This model inherits from [`Mo

Construct the embeddings from word and position embeddings.

Construct the embeddings from word and position embeddings.

An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models.

An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models.

r""" Patch Merging Layer. Args: input_resolution (tuple[int]): Resolution of input feature. dim (int): Number of input channels.

r""" Patch Merging Layer. Args: input_resolution (tuple[int]): Resolution of input feature. dim (int): Number of input channels.

r""" Swin Transformer A PyTorch impl of : `Swin Transformer: Hierarchical Vision Transformer using Shifted Windows` - https://arxiv

r""" A conditional 2D UNet model that takes a noisy sample, conditional state, and a timestep and returns a sample shaped output. This mo