LayerNorm but with an optional bias. PyTorch doesn't support simply bias=False

FilmModule modulation for conditioning.

FilmModule modulation for conditioning.

Subclass torch's LayerNorm to handle fp16.

This wrapper does the following changes to the action and observation: - Flips the image and segmentation observations - Converts the dep

Block with AdaLN-Zero conditioning.

Root mean square normalization layer implementation similar to https://pytorch.org/docs/stable/generated/torch.nn.RMSNorm.html. It normalize

r""" Tri-Stage Learning Rate Scheduler. Implement the learning rate scheduler in "SpecAugment" Similar to inverse_squre_root scheduler, but t

Dataclass to store all chunk size related params for the XL chunk kernel.

Block with AdaLN-Zero conditioning.

Randomly sample crops at input, and then average across crop features at output.

DPM-Solver. See https://arxiv.org/abs/2206.00927.

Block with AdaLN-Zero conditioning.

AdaLN-Zero modulation for conditioning.

AdaLN-Zero modulation for conditioning.

AdaLN-Zero modulation for conditioning.

AdaLN-Zero modulation for conditioning.

A wrapper around torchsde.BrownianTree that enables batches of entropy.

A noise sampler backed by a torchsde.BrownianTree. Args: x (Tensor): The tensor whose shape, device and dtype to use to generate

Block with AdaLN-Zero conditioning.

Block with AdaLN-Zero conditioning.

Maintains (exponential) moving average of a set of parameters.

Gaussian random features for encoding time steps.

Min Max scaler, that scales the output data between -1 and 1 and the input data between 0 and 1.

A ResNet class that is similar to torchvision's but contains the following changes: - There are now 3 "stem" convolutions as opposed to 1, wi

Multi-head version of the LayerNorm layer. It normalizes the last dimension of the input tensor. The input is assumed to have the shape

A PID controller for ODE adaptive step size control.

Perceiver Attention Layer

mLSTM Backend Module for PyTorch. This module wraps the mLSTM kernels and provides a high-level interface for training and inference.