@poses: wrt. the centered mesh

Project the points and find the cropping transform @pts: (N,3) @poses: (B,4,4) tensor @min_box: min_box/min_circle @scale: scale to apply to t

Compute the loss for the FrontTracker model.

construct a dense grid based on resolution

3x3 convolution with padding

Convert the torso of the model to bfloat16.

Convert the torso of the model to float16.

Custom interpolate to avoid INT_MAX issues in nn.functional.interpolate.

Custom interpolate to avoid INT_MAX issues in nn.functional.interpolate.

Decode the structured latent. Args: slat (sp.SparseTensor): The structured latent. formats (List[str]): The

Encode the image. Args: image (Union[torch.Tensor, list[Image.Image]]): The image to encode Returns:

world_tracks: B T N 3 this is the coarse 3d tracks in world coordinate (coarse 3d tracks) dyn_prob: B T N this is the dynamic probabilit

Find foreground-background relations between neighboring pixels. Args: ---- d (np.ndarray): Depth matrix. t (float): Threshol

Run SuperPoint (optionally) and SuperGlue SuperPoint is skipped if ['keypoints0', 'keypoints1'] exist in input Args: data:

Positionally encode points that are not normalized to [0,1].

Load a pretrained model. Args: path (str): The path to the model. Can be either local path or a Hugging Face repository.

This function generates a 1D positional embedding from a given grid using sine and cosine functions. Args: - embed_dim: The embedding di

this will perform the index select, cat the tensors, and provide the attn_bias from cache

this will perform the index select, cat the tensors, and provide the attn_bias from cache

this will perform the index select, cat the tensors, and provide the attn_bias from cache

Get the conditioning information for the model. Args: image (Union[torch.Tensor, list[Image.Image]]): The image prompts.

Returns parameters of all the ModuleDicts children that are exclusively used for the given bin configuration

Returns the positional encoding used to encode point prompts, applied to a dense set of points the shape of the image encoding.

Process input batch data for training. Args: batch_data (dict): Batch data containing images and targets Return

vis_gt: [B, T, N] 0/1 binary tensor n: [B, N] int tensor, the n-th visible time index to get (1-based) Returns: [B, N] tensor of time in

Generate thresholds and weights for the given range. Args: ---- t_min (float): Minimum threshold. t_max (float): Maximum thre

Extract video name without extension

Extract video name without extension

Calculate element-wise Huber loss between x and y

Infer depth and fov for a given image. If the image is not at network resolution, it is resized to 1536x1536 and the estimated depth

This method is called the the thread is started, to initialize any resources that is only held in the thread.

Inject a sampler with multiple images as condition. Args: sampler_name (str): The name of the sampler to inject.

Returns field of view in radians from normalized intrinsics matrix. ### Parameters: - intrinsics: torch.Tensor of shape (..., 3, 3)

cam_pts: (B, T, N, 3) graph_matrix: (B, 1, N) dyn_weight: (B, T, N) K: number of keyframes to select (including start and end) R

Load a PNG file as a mask and its palette.

Load state_dict into model, handling DataParallel and DistributedDataParallel. Also checks for "model" key in state_dict. DataParallel prefixes s

Loads weights to the model from a given resource. A resource can be of following types: 1. URL. Prefixed with "url::" e.g. url

Compute the loss of 3D tracking problem

This function generates a 1D positional embedding from a given grid using sine and cosine functions. Args: - embed_dim: The embedding di

This function generates a 1D positional embedding from a given grid using sine and cosine functions. Args: - embed_dim: The embedding di

Encodes masks to an uncompressed RLE, in the format expected by pycoco tools.

@pose_offset: optimized delta pose of the first frame. Usually it's identity