This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he

This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he

Sample a chunk from the chunked videos. Our videos are stored as regular mp4 files but with chunked captions. There is one caption per [chunk_siz

Split input tensor along the sequence dimension for checkpoint parallelism. This function divides the input tensor into equal parts along th

Given the camera pose, transform input 3D points from camera coordinates to world coordinates. Args: points (torch.Tensor/np.ndar

Check robustly if normalized word and the matching target have a difference of up to guardrail_partial_match_letter_count characters.

Clean up server resources and shutdown workers.

Copy a file src to dst and return the destination file. If dst specifies a file that already exists, it will be replaced. Args:

Check if an object exists in S3 with retry logic.

Convert (x,y,z,w, tx,ty,tz) to world2cam extrinsic matrix/matrices [R|t]. Args: qxyzw_t (torch.Tensor/np.ndarray [...,7]): Quater

Generate samples from the batch. Based on given batch, it will automatically determine whether to generate image or video samples. Ar

Get the batch size from a data batch, a (possibly hierarchical) dictionary of tensors. Args: data (dict[str, torch.Tensor]): Data batch (

Check whether a file path is a directory. Args: filepath (str or Path): Path to be checked whether it is a direct

Check whether a file path is a file. Args: filepath (str or Path): Path to be checked whether it is a file. Returns:

Load network weights and optimizer states from a checkpoint in a single process. The priority of the checkpoint loading logic is: 1.

Load model from checkpoint with optional multi-adapter support. Args: experiment_name: experiment name s3_checkpoint_dir: s3

Check whether an object exists in the storage, with retry logic for transient errors. Args: key (str): The key of the ob

Hook after loss.backward() is called. This method is called immediately after the backward pass, allowing for custom operations or mo

Prepares an embedded sequence tensor by applying positional embeddings and handling padding masks. Args: x_B_C_T_H_W (to

Perform a first-order Runge-Kutta (Euler) step. Recommended for diffusion models with guidance or model undertrained Usually more stable

Saves a Config object to a file using YAML serialization. This method is beneficial when the configuration object's content needs to be human

signal individual worker status per rank Args: rank (int): The rank of the worker status (str): The status of the wor

Copies tensors in data_batch to the GPU and broadcasts across CP or TP ranks. The contents of data_batch are updated with the copied and broadcas

Decode latents using compiled or eager VAE.

Create a layer of residual blocks. - planes: the number of output channels for this layer - blocks: the number of residual bl

Normalizes video data in-place on a CUDA device to reduce data loading overhead. This function modifies the video data tensor within

Parse string to a boolean.

Store an exception from the background thread with context information. Args: exception (Exception): The exception that was raise

Apply AdaLN: norm(x) * (1 + scale) + shift. Assumes tensors are broadcast-compatible with layout (b, t, h, w, d).

Given the camera intrinsics, calibrate input 3D points from camera frame to image (pixel) frame. Args: points (torch.Tensor/np.nd

Stop the prefetch thread and clear the cache. Also checks for any errors in the background thread and raises them.

Smartly concatenate n input data batches into m output data batches. Each data batch is a dictionary with values that can be torch tensor, st

Updates nested dictionaries in the config dictionary (dst) with the values in src dictionary. Standard update in hydra only goes one level de