A sampler wrapper for grouping images with similar aspect ratio into a same batch. Args: sampler (Sampler): Base sampler. dataset

2D Image to Patch Embedding

Multi-head Attention block with KV token compression and qk norm.

Embeds class labels into vector representations. Also handles label dropout for classifier-free guidance.

This debug class helps detect and understand where the model starts getting very large or very small, and more importantly `nan` or `inf` wei

Embeds scalar timesteps into vector representations.

The final layer of PixArt.

Utilities for training and sampling diffusion models. Original ported from this codebase: https://github.com/hojonathanho/diffusion/blob/

`LCMScheduler` extends the denoising procedure introduced in denoising diffusion probabilistic models (DDPMs) with non-Markovian guidance.

Output class for the scheduler's `step` function output. Args: prev_sample (`torch.FloatTensor` of shape `(batch_size, num_channels,

Diffusion model with a Transformer backbone.

A PixArt block with adaptive layer norm (adaLN-single) conditioning.

Diffusion model with a Transformer backbone.

A PixArt block with adaptive layer norm zero (adaLN-Zero) conditioning.

A diffusion process which can skip steps in a base diffusion process. :param use_timesteps: a collection (sequence or set) of timesteps from

Embeds scalar timesteps into vector representations.

Embeds class labels into vector representations. Also handles label dropout for classifier-free guidance.

The final layer of PixArt.

The final layer of PixArt.

Gather tensors from all process, supporting backward propagation.

Gather tensors from all process, supporting backward propagation.

Embeds class labels into vector representations. Also handles label dropout for classifier-free guidance.

Redirect stderr to stdout, optionally print stdout to a file, and optionally force flushing on both stdout and the file.

The final layer of PixArt.

Which type of output the model predicts.

What is used as the model's output variance. The LEARNED_RANGE option has been added to allow the model to predict values between FIXED_S

Multi-Query self attention. Using torch or triton attention implemetation enables user to also use additive bias.

Multi-head self attention. Using torch or triton attention implemetation enables user to also use additive bias.

r""" tmp Pipeline for text-to-image generation using PixArt-Sigma.

`SASolverScheduler` is a fast dedicated high-order solver for diffusion SDEs. This model inherits from [`SchedulerMixin`] and [`ConfigMixin`

A distribution over timesteps in the diffusion process, intended to reduce variance of the objective. By default, samplers perform unbias