img if of size batch_size x 3 x h x w

input: * tokens: batch_size x nheads x ntokens x dim * positions: batch_size x ntokens x 2 (y and x posit

Forward pass. Args: x (tensor): input Returns: tensor: output

Forward pass. Returns: tensor: output

Forward pass. Args: x (tensor): input Returns: tensor: interpolated data

Backprojects spatial points in 2D image space to world space using invK_b44 at the depths defined in depth_b1hw.

Projects spatial points in 3D world space to camera image space using the extrinsics matrix cam_T_world_b44 and intrinsics K_b44.

First smoothes incoming depth maps with a gaussian blur, backprojects those depth points into world space (see BackprojectDepth), e

Return a list of images corresponding to a spiral trajectory from a random starting point. Useful to generate nice visualisations.

Apply a geometric transformation to a list of 3-D points. H: 3x3 or 4x4 projection matrix (typically a Homography) p: numpy/torch/tuple of c

grid_size: int of the grid height and width return: pos_embed: [grid_size*grid_size, embed_dim] or [n_cls_token+grid_size*grid_size, embe

Computes camera rays for given camera data and points, optionally shifts rays to camera frame.

Normalizes an image with ImageNet statistics.

ViT weight initialization, original timm impl (for reproducibility)

Invert a torch or numpy matrix

Loads images from a directory or video, resizes them to a uniform size, then converts and stacks them into a single [N, 3, H, W] PyTorch tens

Logs camera intri/extri, depth, rgb, and mesh to rerun.

imgs: (B, 3, H, W) x: (B, L, patch_size**2 *3)

DVMVS frame pose distance.

to be defined for each dataset

to be defined for each dataset

Helper function for quickly exporting depth maps during inference.

Quaternion to 3x3 rotation matrix.

readFlowFile(<FILENAME>) reads a flow file <FILENAME> into a 2-band np.array. if <FILENAME> does not exist, an IOError is raised. if <FIL

Reads an image file using PIL, then optionally resizes the image, with selective resampling, scales values, and returns the image as a tenso

Reads in a text file and returns lines in a list.

3D Rotation about the x-axis.

3D Rotation about the y-axis.

3D Rotation about the z-axis.

Saves a video file of numpy RGB frames in frame_list.

Computes the inverse of a batch of SE(3) matrices. T: Tensor of shape (B, 4, 4)

Removes the 'module.' prefix from the keys of a state_dict. Args: state_dict (dict): The original state_dict with possible 'module.'

Removes the 'module.' prefix from the keys of a state_dict. Args: state_dict (dict): The original state_dict with possible 'module.'

Synchronize the count and total across all processes.

Unpacks a flattened tensor of tupled elements (BS) into bMS. Tuple size is M.

Packs an inflated tensor of tupled elements (bMS) into BS. Tuple size is M.

Converts a PyTorch tensor to a PIL image. Automatically moves the channel dimension (if it has size 3) to the last axis before converting.

" Moves tensors in the input dict to the gpu and ignores tensors/elements as with keys in key_ignores.

Moves tensors in the input dict to the gpu and ignores tensors/elements as with keys in key_ignores.

x: (N, L, patch_size**2 *channels) imgs: (N, 3, H, W)

Upsample input tensor by a factor of 2

Close to ViT-giant, with embed-dim 1536 and 24 heads => embed-dim per head 64