Extract CLIP features from videos.

Extract DINO features from the video.

Forward pass. Returns: - dec: reconstructed input, uses mode of latent distribution if sample_posterior is False, otherwi

Standard forward.

Standard forward.

Update video metrics with the given predictions and targets. Args: preds: Predictions of shape (B, T, C, H, W), [0, 1]

Compute the cosine similarity score. It is the average of: - the cosine similarity between consecutive frames. - the

Compute the aesthetic quality score. A 0-10 rating is assigned to each frame by applying LAION aesthetic predictor to CLIP features,

Compute the temporal flickering score, which is the average of MAE between consecutive frames. MAE is then normalized from [0, 255] t

Compute the dynamic degree score, which is 0 or 1 depending on whether the video is static or non-static (decided by the optical flow from RA

Compute the imaging quality score. A 0-100 rating is assigned to each frame using MUSIQ image quality predictor, then normalized to [

Compute the dimension score. Args: videos: Videos of shape (B, T, C, H, W), uint8, range [0, 255]. Returns:

Compute the motion smoothness, which is computed by: 1. Drop odd frames and interpolate them by passing the even frames through AMT-S

Args: videos (torch.Tensor): (B, S, C, H, W), range [-1, 1]

Estimate optical flow between pair of frames

Given the timestep t within [0, 1], return the logSNR value at that timestep.

Args: x: a [... x N x ... x D] if flatten=False, [... x (N x ...) x D] if flatten=True tensor of queries or keys. Returns

Randomly nullify the input embeddings with a probability p during training. For inference, the embeddings are nullified only if mask is provi

Args: x: tensor to be patchified of shape (*B, C, H, W) Returns: patchified tensor of shape (*B, num_patches,

Forward pass through the RMSNorm layer. Args: x (torch.Tensor): The input tensor. Returns: torch.Te

Forward pass of the U-ViT backbone, with pose conditioning. Args: x: Input tensor of shape (B, T, C, H, W). n