Apply Non-Maximum Suppression (NMS) vertically on the given ratio matrix. Args: ---- ratio (np.ndarray): Input ratio matrix.

Normalize keypoints locations based on image image_shape

UV with left-top corner as (-width / diagonal, -height / diagonal) and right-bottom corner as (width / diagonal, height / diagonal)

UV with left-top corner as (-width / diagonal, -height / diagonal) and right-bottom corner as (width / diagonal, height / diagonal)

Training loss calculation

Patchify a tensor. Args: x (torch.Tensor): (N, C, *spatial) tensor patch_size (int): Patch size

This function convert the pose encoding into `intrinsic` and `extrinsic` Args: poses_pred: B T 8 Return: Intrinsic B T

x_t = (1 - t) * x0 + t * noise

Transforms model's predictions to semantically meaningful values. Args: raw: [num_rays, num_samples along ray, 4]. Prediction from model.

Octree is in normalized [-1,1] world coordinate frame 'rays_o': ray origin in normalized world coordinate system 'rays_d': (N,3) unit length r

Read a depth image, return float32 depth array of shape (H, W).

Read a image, return uint8 RGB array of shape (H, W, 3).

Removes small disconnected regions and holes in a mask. Returns the mask and an indicator of if the mask has been modified.

Render rays Args: H: int. Height of image in pixels. W: int. Width of image in pixels. K: float. Focal length of pinhole camera.

Render the octree. Args: octree (Octree): octree extrinsics (torch.Tensor): (4, 4) camera extrinsics

Right shift an array of binary values. Parameters: ----------- binary: An ndarray of binary values. k: The number of bits to shift

Compute a binary mask from an uncompressed RLE.

Generate sparse structure from an input image. Args: image (Image.Image): Input image num_samples (int): Num

Run inference with either original SAM or HF SAM predictor Args: predictor: SamPredictor or HFSamPredictor instance input_x: Input

Run the pipeline with multiple images as condition Args: images (List[Image.Image]): The multi-view images of the assets

Generate samples from the model using Euler method. Args: model: The model to sample from. noise: Th

Sample from a model.

Generate samples from the model using Euler method. Args: model: The model to sample from. noise: Th

r"""Sample spatial features `sample_features4d(input, coords)` samples the spatial features :attr:`input` represented by a 4D tensor :math:`(

We first connect the discontinuous boxes for each ray and treat it as uniform sample, then we disconnect into correct boxes @rays_d: (N_ray,3)

Sample structured latent with the given conditioning. Args: cond (dict): The conditioning information.

Sample structured latent with the given conditioning. Args: cond (dict): The conditioning information.

Sample sparse structures with the given conditioning. Args: cond (dict): The conditioning information.

Save a mask as a PNG file with the given palette.

Returns the bounding box of the scene. Taken from Shap-E rendering script (https://github.com/openai/shap-e/blob/main/shap_e/rendering/blende

Get the context for the attention scaled dot-product kernel. We use Flash Attention by default, but fall back to all available kernels if Fla

Calculates the image embeddings for the provided image, allowing masks to be predicted with the 'predict' method. Arguments:

Processes specialized tokens with shape (1, 2, X, C) for multi-frame processing: 1) Uses the first position (index=0) for the first frame onl

Solve `min |focal * xy / (z + shift) - uv|` with respect to shift and focal

Solve `min |focal * xy / (z + shift) - uv|` with respect to shift and focal

Solve `min |focal * xy / (z + shift) - uv|` with respect to shift

Solve `min |focal * xy / (z + shift) - uv|` with respect to shift

Broadcast a 1D tensor to a sparse tensor along the batch dimension then perform an operation. Args: input (torch.Tensor): 1D ten

Broadcast a 1D tensor to a sparse tensor along the batch dimension then perform an operation. Args: input (torch.Tensor): 1D ten

Apply scaled dot product attention to a sparse tensor. Args: qkv (SparseTensor): A [N, *, 3, H, C] sparse tensor containing Qs, Ks,

Apply serialized scaled dot product self attention to a sparse tensor. Args: qkv (SparseTensor): [N, *, 3, H, C] sparse tensor conta

Apply windowed scaled dot product self attention to a sparse tensor. Args: qkv (SparseTensor): [N, *, 3, H, C] sparse tensor contain

Convert the mesh to a trimesh.Trimesh object. Args: transform_pose (bool): If True, transform the vertices to change coor

Transform the batch before feeding to the network !NOTE the H_ori, W_ori could be different at test time from the training data, and needs to be s

Unpatchify a tensor. Args: x (torch.Tensor): (N, C, *spatial) tensor patch_size (int): Patch size

args: rgbs: [S C H W] R: [S 3 3] T: [S 3] xy_depth: [S H W 3] focal_length: [S] pick_idx: list of

split the video into several overlapped windows. args: x: the input video frames. [B, T, C, H, W] outp

Build the patch embed layer. If 'conv', we use a simple PatchEmbed conv layer. Otherwise, we use a vision transformer.