Process a single image with given instruction Args: image (PIL.Image): Input image instruction (str)

Process a single image with given instruction Args: image (PIL.Image): Input image instruction (str)

Process a single image with given instruction Args: image (PIL.Image): Input image instruction (str)

Transforms 3D points to 2D using extrinsic and intrinsic parameters. Args: points3D (torch.Tensor): 3D points of shape Px3. e

Quaternion Order: XYZW or say ijkr, scalar-last Convert rotations given as quaternions to rotation matrices. Args: quaternions:

Refines the tracking of images using a fine track predictor and a fine feature network. Check https://arxiv.org/abs/2312.04563 for more detai

Route task based on instruction

Route task based on instruction

Forward pass through the fusion blocks. Args: features (List[Tensor]): List of feature maps from different layers.

Segment objects in an image based on a query Args: image: Input image query: Text query describing w

Apply undistortion to the normalized tracks using the given distortion parameters once. Args: params (torch.Tensor or numpy.ndarray)

Convert a unit quaternion to a standard form: one in which the real part is non negative. Args: quaternions: Quaternions with re

Iteratively refine camera pose predictions. Args: pose_tokens (torch.Tensor): Normalized camera tokens with shape [B, 1,

Create a video for depth estimation with three panels shown sequentially

Create a video for pose estimation with three panels shown sequentially

Enhanced visualization with three-panel layout

Create a video with three panels shown sequentially, no white borders

Generates spatial positions for a batch of patches. Args: batch_size: Number of samples in the batch. height: Height

Initialize the QwenVLCoT model Args: model_path (str): Path to the model

Initialize the VisionReasoner model with reasoning and segmentation components Args: reasoning_model_path (str):

Initialize the VisionReasoner model with reasoning and segmentation components Args: reasoning_model_path (str):

Initialize the QwenVL model Args: model_path (str): Path to the model

Initialize task router

Initializes the 2D RoPE module.

Args: img_size (int, tuple): input image size patch_size (int, tuple): patch size in_chans (int): number

Init. Args: features (int): number of features

Init. Args: features (int): number of features

Initialize the TrackHead module. Args: dim_in (int): Input dimension of tokens from the backbone. patch_size

Self attention block

Cross attention block

Self attention block

Cross attention block

Answer a question about an image Args: image: Input image question: Text question

Answer a question about an image Args: image: Input image question: Text question

Answer questions about a batch of images Args: images: List of input images questions: List of text

Answer questions about a batch of images Args: images: List of input images questions: List of text

Answer questions about a batch of images Args: images: List of input images questions: List of text

Answer questions about a batch of images Args: images: List of input images questions: List of text

Answer questions about a batch of images Args: images: List of input images questions: List of text

Convert Batched NumPy Arrays to PyCOLMAP Check https://github.com/colmap/pycolmap for more details about its format NOTE that colmap ex

Convert Batched NumPy Arrays to PyCOLMAP Different from batch_np_matrix_to_pycolmap, this function does not use tracks. It saves points

Normalize predicted tracks based on camera intrinsics. Args: intrinsics (torch.Tensor): The camera intrinsics tensor of shape [batch_size

Calculate IOU matrix between two sets of bounding boxes bboxes1: shape (N, 4) prediction boxes bboxes2: shape (M, 4) ground truth boxes

Count objects in an image based on a query Args: image: Input image query: Text query describing wha

Count objects in a batch of images Args: images: List of input images queries: List of text queries

Count objects in a batch of images Args: images: List of input images queries: List of text queries

Count objects in a batch of images Args: images: List of input images queries: List of text queries

Count objects in a batch of images Args: images: List of input images queries: List of text queries

Creates a grid of pixel coordinates and frame indices for all frames. Returns: tuple: A tuple containing: - points_xyf (n

Detect objects in an image based on a query Args: image: Input image query: Text query describing wh

Detect objects in a batch of images Args: images: List of input images queries: List of text queries

Detect objects in a batch of images Args: images: List of input images queries: List of text queries

Detect objects in a batch of images Args: images: List of input images queries: List of text queries

Detect objects in a batch of images Args: images: List of input images queries: List of text queries

Convert camera extrinsics and intrinsics to a compact pose encoding. This function transforms camera parameters into a unified pose encoding form

Applies 2D rotary position embeddings to input tokens. Args: tokens: Input tensor of shape (batch_size, n_heads, n_tokens, dim).