Create a 1D, 2D, or 3D average pooling module.

Compute the entire variational lower-bound, measured in bits-per-dim, as well as other related quantities. :param model: t

Center cropping implementation from ADM. https://github.com/openai/guided-diffusion/blob/8fb3ad9197f16bbc40620447b2742e13458d2831/guided_di

Evaluate a function without caching intermediate activations, allowing for reduced memory at the expense of extra compute in the backward p

End DDP training.

r""" Copy from torch.nn.utils.clip_grad_norm_ Clips gradient norm of an iterable of parameters. The norm is computed over all gradients

Compute the log-likelihood of a continuous Gaussian distribution. :param x: the targets :param means: the Gaussian mean Tensor. :

Create a 1D, 2D, or 3D convolution module.

A counter for the `thop` package to count the operations in an attention operation. Meant to be used like: macs, params = tho

Create a logger that writes to a log file and stdout.

Create a logger that writes to a log file and stdout.

Create a ScheduleSampler from a library of pre-defined samplers. :param name: the name of the sampler. :param diffusion: the diffusion

Create a tensorboard that saves losses.

Sample x_{t+1} from the model using DDIM reverse ODE.

Forward pass of DiT, but also batches the unconditional forward pass for classifier-free guidance.

r""" Copy from torch.nn.utils.clip_grad_norm_ Clips gradient norm of an iterable of parameters. The norm is computed over all gradients

Get a pre-defined beta schedule for the given name. The beta schedule library consists of beta schedules which remain similar in the l

Args: clip (torch.tensor): Video clip to be normalized. Size is (T, C, H, W) Returns: flipped clip (torch.tensor): Size i

Create a linear module.

Take the mean over all non-batch dimensions.

Make a standard normalization layer. :param channels: number of input channels. :return: an nn.Module for normalization.

Slide along the long edge, with the short edge as crop size

Set requires_grad flag for all parameters in a model.

Do spatial cropping and resizing to the video clip Args: clip (torch.tensor): Video clip to be cropped. Size is (T, C, H, W)

Scale the parameters of a module and return it.

separate coontent and motion in a given video Args: video_clip, a give video clip, [B F C H W] Return: base frame, [B, 1

Initialize distributed training environment. support both slurm and torch.distributed.launch see torch.distributed.init_process_group() for mo

Create sinusoidal timestep embeddings. :param timesteps: a 1-D Tensor of N indices, one per batch element. These may

Compute training losses for a single timestep. :param model: the model to evaluate loss on. :param x_start: the [N x C x .

Step the EMA model towards the current model.

Update the reweighting using losses from a model. Call this method from each rank with a batch of timesteps and the corres