Load the video frames from video_path. The frames are resized to image_size as in the model and are loaded to GPU if offload_video_to_cpu=Fal

Load the video frames from a directory of JPEG files ("<frame_index>.jpg" format). The frames are resized to image_size x image_size and are

Load the video frames from a video file.

compute bounding box given an input mask Inputs: - masks: [B, 1, H, W] masks, dtype=torch.Tensor Returns: - box_coords: [B, 1,

Perform PostProcessing on output masks.

Predicts masks. See 'forward' for more details.

Prepare inference_state and consolidate temporary outputs before tracking.

Args: img: same as in __init__

Remove all input points or mask in all frames throughout the video.

Sample 1 random point (along with its label) from the center of each error region, that is, the point with the largest distance to the bounda

Sample `num_pt` random points (along with their labels) independently from the error regions. Inputs: - gt_masks: [B, 1, H_im, W_im] mas

Select up to `max_cond_frame_num` conditioning frames from `cond_frame_outputs` that are temporally closest to the current frame at `frame_id

Calculates the image embeddings for the provided image, allowing masks to be predicted with the 'predict' method. Arguments:

calculate the number of frames for video used for model inputs. Args: ele (dict): a dict contains the configuration of video.

Expects a tensor of shape Bx4. The coordinates can be in absolute image or normalized coordinates, if the coords are in absolute imag

Partition into non-overlapping windows with padding if needed. Args: x (tensor): input tokens with [B, H, W, C]. window_size

Window unpartition into original sequences and removing padding. Args: x (tensor): input tokens with [B * num_windows, window_size, w

Using a SAM 2 model, generates masks for the entire image. Generates a grid of point prompts over the image, then filters low

Uses SAM-2 to calculate the image embedding for an image, and then allow repeated, efficient mask prediction given prompts.

This class computes the multi-step multi-mask and IoU losses. Args: weight_dict: dict containing weights for focal, dice,

Args: kernel_size (Tuple): kernel size of the projection layer. stride (Tuple): stride of the projection layer.

Predicts masks given an image and prompt embeddings, using a transformer architecture. Arguments: transformer_dim

A transformer decoder that attends to an input image using queries whose positional embedding is supplied. Args: d

A transformer block with four layers: (1) self-attention of sparse inputs, (2) cross attention of sparse inputs to dense inputs, (3)

Encodes prompts for input to SAM's mask decoder. Arguments: embed_dim (int): The prompts' embedding dimension im

Transforms for SAM2.

Args: img (ndarray): an RGB image of shape (H, W, 3) in range [0, 255]. scale (float): scale the input image

Args: img_rgb: a numpy array of shape (H, W, C), where H and W correspond to the height and width of the image re