r""" Norm layer modified to incorporate timestep embeddings. Parameters: embedding_dim (`int`): The size of each embedding vector.

this is partly inspired by vllm

r""" A cross attention layer. Parameters: query_dim (`int`): The number of channels in the query. cross_attention

r"""A 3D causal convolution layer that pads the input tensor to ensure causality in CogVideoX Model. Args: in_channels (`int`): Number of

r""" A feed-forward layer. Parameters: dim (`int`): The number of channels in the input. dim_out (`int`, *optional*): The num

r""" A cross attention layer. Parameters: query_dim (`int`): The number of channels in the query. cross_attention

r""" A 3D convolution layer that splits the input tensor into smaller parts to avoid OOM in CogVideoX Model.

A 2D Transformer model for image-like data. Parameters: num_attention_heads (`int`, *optional*, defaults to 16): The number of heads

r""" Processor for implementing scaled dot-product attention (enabled by default if you're using PyTorch 2.0).

r""" Processor for implementing scaled dot-product attention (enabled by default if you're using PyTorch 2.0).

This config is to instantiate a `CogVideoXPipeline` class for video generation. To be specific, this config will be passed to engine by `Vid

r""" A 3D ResNet block used in the CogVideoX model. Args: in_channels (`int`): Number of input channels. out_chan

r""" Spatially conditioned normalization as defined in https://arxiv.org/abs/2209.09002. This implementation is specific to 3D-video like data

r""" A feed-forward layer. Parameters: dim (`int`): The number of channels in the input. dim_out (`int`, *optional*): The num

r""" A feed-forward layer. Parameters: dim (`int`): The number of channels in the input. dim_out (`int`, *optional*): The num

r""" A gated self-attention dense layer that combines visual features and object features. Parameters: query_dim (`int`): The number

This config is to instantiate a `LattePipeline` class for video generation. To be specific, this config will be passed to engine by `VideoSy

This config is to instantiate a `OpenSoraPipeline` class for video generation. To be specific, this config will be passed to engine by `Vide

This config is to instantiate a `OpenSoraPlanPipeline` class for video generation. To be specific, this config will be passed to engine by `

2D Image to Patch Embedding

r""" Norm layer adaptive layer norm single (adaLN-single). As proposed in PixArt-Alpha (see: https://arxiv.org/abs/2310.00426; Section 2.3).

r""" A middle block used in the CogVideoX model. Args: in_channels (`int`): Number of input channels. temb_channe

r""" A feed-forward layer. Parameters: dim (`int`): The number of channels in the input. dim_out (`int`, *optional*): The num

r""" A gated self-attention dense layer that combines visual features and object features. Parameters: query_dim (`int`): The number

r""" A gated self-attention dense layer that combines visual features and object features. Parameters: query_dim (`int`): The number

Result of task dispatched to worker

Convert tensor data type from uint8 to float, divide value by 255.0 and permute the dimensions of clip tensor

The output of [`Transformer2DModel`]. Args: sample (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)` or `(ba

The output of [`Transformer2DModel`]. Args: sample (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)` or `(ba

r""" Norm layer adaptive layer norm single (adaLN-single). As proposed in PixArt-Alpha (see: https://arxiv.org/abs/2310.00426; Section 2.3).

r""" Norm layer adaptive layer norm single (adaLN-single). As proposed in PixArt-Alpha (see: https://arxiv.org/abs/2310.00426; Section 2.3).

Compatible with old versions, there are issues, use with caution.

Compatible with old versions, there are issues, use with caution.

r""" Processor for implementing scaled dot-product attention (enabled by default if you're using PyTorch 2.0).

r""" A basic Transformer block. Parameters: dim (`int`): The number of channels in the input and output. num_attention_heads

r""" A basic Transformer block. Parameters: dim (`int`): The number of channels in the input and output. num_attention_heads

r""" A basic Transformer block. Parameters: dim (`int`): The number of channels in the input and output. num_attention_heads

r""" A basic Transformer block. Parameters: dim (`int`): The number of channels in the input and output. num_attention_heads

r""" A basic Transformer block. Parameters: dim (`int`): The number of channels in the input and output. num_attention_heads

r""" A basic Transformer block. Parameters: dim (`int`): The number of channels in the input and output. num_attention_heads

r""" A basic Transformer block. Parameters: dim (`int`): The number of channels in the input and output. num_attention_heads

Projects caption embeddings. Also handles dropout for classifier-free guidance. Adapted from https://github.com/PixArt-alpha/PixArt-alpha/bl

Projects caption embeddings. Also handles dropout for classifier-free guidance. Adapted from https://github.com/PixArt-alpha/PixArt-alpha/bl

r""" Processor for implementing scaled dot-product attention for the CogVideoX model. It applies a rotary embedding on query and key vectors,

r""" Transformer block used in [CogVideoX](https://github.com/THUDM/CogVideo) model. Parameters: dim (`int`): The number

r""" The `CogVideoXDecoder3D` layer of a variational autoencoder that decodes its latent representation into an output sample. Args:

r""" A downsampling block used in the CogVideoX model. Args: in_channels (`int`): Number of input channels. out_c

r""" A 3D Downsampling layer using in [CogVideoX]() by Tsinghua University & ZhipuAI Args: in_channels (`int`): Number of

r""" The `CogVideoXEncoder3D` layer of a variational autoencoder that encodes its input into a latent representation. Args: in_channe

r""" An upsampling block used in the CogVideoX model. Args: in_channels (`int`): Number of input channels. out_ch

r""" A 3D Upsample layer using in CogVideoX by Tsinghua University & ZhipuAI # Todo: Wait for paper relase. Args: in_channels (`int`)

For PixArt-Alpha. Reference: https://github.com/PixArt-alpha/PixArt-alpha/blob/0f55e922376d8b797edd44d25d0e7464b260dcab/diffusion/model/

For PixArt-Alpha. Reference: https://github.com/PixArt-alpha/PixArt-alpha/blob/0f55e922376d8b797edd44d25d0e7464b260dcab/diffusion/model/

Output class for the scheduler's `step` function output. Args: prev_sample (`torch.Tensor` of shape `(batch_size, num_channels, heig

Output class for the scheduler's `step` function output. Args: prev_sample (`torch.Tensor` of shape `(batch_size, num_channels, heig

Decoder Blocks.

Encoder Blocks.

r""" A Transformer block following the MMDiT architecture, introduced in Stable Diffusion 3. Reference: https://arxiv.org/abs/2403.03206

to match AttnBlock usage

to match AttnBlock usage

Code from https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L148

Code from https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L148

Code from https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L148

Code from https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L148

Embeds class labels into vector representations. Also handles label dropout for classifier-free guidance.