Compute the entire variational lower-bound, measured in bits-per-dim, as well as other related quantities. :param model: the

End DDP training.

Compute the log-likelihood of a continuous Gaussian distribution. :param x: the targets :param means: the Gaussian mean Tensor. :para

Create a logger that writes to a log file and stdout.

Create a ScheduleSampler from a library of pre-defined samplers. :param name: the name of the sampler. :param diffusion: the diffusion ob

Create a tensorboard that saves losses.

Sample x_{t+1} from the model using DDIM reverse ODE.

Generate samples from the model using DDIM. Same usage as p_sample_loop().

Evaluate if new feature vectors are at the manifold.

head_select: (b, num_layers, num_head)

r""" The forward method of the `Attention` class. Args: hidden_states (`torch.Tensor`): The hidden states

Forward pass of DiT, but also batches the unconditional forward pass for classifier-free guidance.

r""" Copy from torch.nn.utils.clip_grad_norm_ Clips gradient norm of an iterable of parameters. The norm is computed over all gradients

Get a pre-defined beta schedule for the given name. The beta schedule library consists of beta schedules which remain similar in the limi

r""" Get the attention processor in use. Args: return_deprecated_lora (`bool`, *optional*, defaults to `False`):

Set requires_grad flag for all parameters in a model.

r""" Set the slice size for attention computation. Args: slice_size (`int`): The slice size for attention

r""" Set whether to use memory efficient attention from `xformers` or not. Args: use_memory_efficient_attention_xformers

r""" Set whether to use npu flash attention from `torch_npu` or not.

Initialize distributed training environment. support both slurm and torch.distributed.launch see torch.distributed.init_process_group() for mo

Warning: does not synchronize the deque!

Compute training losses for a single timestep. :param model: the model to evaluate loss on. :param x_start: the [N x C x ...]

Compute training losses for a single timestep. :param model: the model to evaluate loss on. :param x_start: the [N x C x ...]

Step the EMA model towards the current model.

Update the reweighting using losses from a model. Call this method from each rank with a batch of timesteps and the correspon

write the loss information to a tensorboard file. Only for pytorch DDP mode.