Concatenates a list of DiagonalGaussianDistributions along the batch dimension.

Creates a CameraPose object from the raw camera poses. Args: raw_camera_poses (torch.Tensor): The raw camera poses. Shape

Log the history guidance visualization to wandb if logger is provided, or save the visualization as a gif otherwise.

Return the length of the video at idx

Returns the extrinsics matrix [R | T] for the camera poses. Args: flatten (bool): Whether to flatten the extrinsics matri

Extend the tensor by adding dimensions at the end to match x_stacked_shape.

Subsample the dataset with the given filter function

Post-process the conditions before feeding them to the model. For example, conditions that should be computed relatively (e.g. relati

take in video tensors in range [-1, 1] and log into wandb :param observation_gt: ground-truth observation tensor of shape (batch, frame, cha

Return True with probability p

Convert video_path to latent_path

Compute the 4 corners of the camera frustum at the given distance using actual camera intrinsics. Args: poses: C

Run the blocks (except up/downsampling blocks) for a given level, accompanied by reshaping operations before and after.

Initialize the history guidance from a configuration. config.name specifies the name of the history guidance scheme, and the

Modify the input before passing it to the sampling process. Args ---- x: torch.Tensor, Shape (batch_size, seq_len, .

Converts the raw camera poses to concat-flattened intrinsics and extrinsics. Args: external_cond (torch.Tensor): Raw came

Return list of latent paths for the given split

Load video and conditions from video_idx with given start_frame and end_frame (exclusive)

Adapted from torchvision.io.video.read_video Simplified to only read video frames (not audio and additional info) Args: filename

Zero out the parameters of a module and return it.

Args: hidden_size: Number of features in the hidden layer. num_heads: Number of attention heads. mlp_rati

Compute mask indicating if distances are above threshold

Converts the number of frames to the number of tokens. - Chunk-wise VideoVAE: 1st frame -> 1st token, then every self.temporal_downsa

Convert a yaml file to a list of command line arguments.

Compose the scores from different history segments using the weights.

Download a file from DFoT Hugging Face model hub. https://huggingface.co/kiwhansong/DFoT

A history segment that contains the full history.

Gather tensors or collections of tensors from all devices, and stack them along the batch dimension. Args: data:

Returns the intrinsics matrix for the camera poses. Args: flatten (bool): Whether to flatten the intrinsics matrix.

Return a query that is the target of the retrieval

Build the lightning module :return: a pytorch-lightning module to be launched

Deterministically build idx_remap for the dataset, which maps the indices of the current dataset to the absolute indices of the full dataset

Compute frustum overlap using Monte Carlo sampling. Samples points inside each frustum and checks how many are contained in the other

Extend the tensor a by adding dimensions at the end to match the shape of x.

Normalizes so that the camera given by R_ref and T_ref becomes the world coordinates. Args: R_ref (torch.Tensor): The rot

Given a tensor y of shape (B, T, ...), pad it at the end across the time dimension to have a length of self.max_tokens.

Post-processes external condition. Args: external_cond: (T, *) tensor, T = self.n_frames Returns: pro