Forward pass through the DPT head, supports processing by chunking frames. Args: aggregated_tokens_list (List[Tensor]): L

Forward pass. Args: x (tensor): input Returns: tensor: output

Forward pass. Returns: tensor: output

Forward pass to predict camera parameters. Args: aggregated_tokens_list (list): List of token tensors from the network;

Forward pass of the TrackHead. Args: aggregated_tokens_list (list): List of aggregated tokens from the backbone.

query_points: B x N x 2, the number of batches, tracks, and xy fmaps: B x S x C x HH x WW, the number of batches, frames, and feature

Args: images (torch.Tensor): Input images with shape [B, S, 3, H, W], in range [0, 1]. B: batch size, S: sequence

Forward pass of the VGGT model. Args: images (torch.Tensor): Input images with shape [S, 3, H, W] or [B, S, 3, H, W], in

Args: images (torch.Tensor): Images as RGB, in the range of [0, 1], with a shape of B x S x 3 x H x W. query_points (

query_points: B x N x 2, the number of batches, tracks, and xy fmaps: B x S x C x HH x WW, the number of batches, frames, and feature

ViT weight initialization, original timm impl (for reproducibility)

ViT weight initialization, original timm impl (for reproducibility)

Load and preprocess images by center padding to square and resizing to target size. Also returns the position information of original pixels

Convert a pose encoding back to camera extrinsics and intrinsics. This function performs the inverse operation of extri_intri_to_pose_encoding,

Pose estimation in an image based on a query Args: image: Input image query: Text query describing w

Predict tracks for the given images and masks. TODO: support non-square images TODO: support masks This function predicts the trac

Process a batch of images with given instructions Args: batch_images (list): List of PIL Images batc

Process a batch of images and instructions Args: batch_images: List of input images batch_instructio

Process a batch of images with given instructions Args: batch_images (list): List of PIL Images batc

Process a batch of images with given instructions Args: batch_images (list): List of PIL Images batc

Process a batch of images with given instructions Args: batch_images (list): List of PIL Images batc

Process a single image and instruction Args: image: Input image instruction: Text instruction/query

Transforms 3D points to 2D using extrinsic and intrinsic parameters. Args: world_points (torch.Tensor): 3D points of shape Px3.

Transforms 3D points to 2D using extrinsic and intrinsic parameters. Args: world_points (torch.Tensor): 3D points of shape BxSxHxWx3.

Convert a PyCOLMAP Reconstruction Object to batched NumPy arrays. Args: reconstruction (pycolmap.Reconstruction): The reconstruction

If mask has more than max_trues True values, randomly keep only max_trues of them and set the rest to False.

COPIED FROM VGGSfM Refines the tracking of images using a fine track predictor and a fine feature network. Check https://arxiv.org/abs/2

Batch route tasks Args: instructions: List of instructions Returns: list: List of result

Route input instruction to corresponding task category Args: instruction: User input instruction Ret

Segment objects in an image based on a query Args: image: Input image query: Text query describing w

Segment objects in a batch of images Args: images: List of input images queries: List of text querie

Segment objects in a batch of images Args: images: List of input images queries: List of text querie

Segment objects in a batch of images Args: images: List of input images queries: List of text querie

Segment objects in a batch of images Args: images: List of input images queries: List of text querie

Unproject a batch of depth maps to 3D world coordinates. Args: depth_map (np.ndarray): Batch of depth maps of shape (S, H, W, 1) or

保存一行三列的可视化图像

保存一行三列的可视化图像

Visualizes frames in a grid layout with specified frames per row. Each track's color is determined by its (x,y) position in the first vis

Close to ViT-giant, with embed-dim 1536 and 24 heads => embed-dim per head 64