Collects the state dict and dump to disk.

Calculates the image embeddings for the provided image, allowing masks to be predicted with the 'predict' method. Arguments:

Calculates the image embeddings for the provided image, allowing masks to be predicted with the 'predict' method. Expects the input

Args: inputs: A float tensor of arbitrary shape. The predictions for each example. targets: A float tensor with t

Resize tokenizer and embedding. Note: This is the unoptimized version that may make your embedding size not be divisible by 64.

Main training loop.

Partition into non-overlapping windows with padding if needed. Args: x (tensor): input tokens with [B, H, W, C]. window_size

Window unpartition into original sequences and removing padding. Args: windows (tensor): input tokens with [B * num_windows, window_s

Uses SAM to calculate the image embedding for an image, and then allow repeated, efficient mask prediction given prompts. Ar

Using a SAM model, generates masks for the entire image. Generates a grid of point prompts over the image, then filters low q

Predicts masks given an image and prompt embeddings, using a transformer architecture. Arguments: transformer_dim

Args: img_size (int): Input image size. patch_size (int): Patch size. in_chans (int): Number of input ima

Args: dim (int): Number of input channels. num_heads (int): Number of attention heads. qkv_bias (bool):

Args: kernel_size (Tuple): kernel size of the projection layer. stride (Tuple): stride of the projection layer.

A transformer decoder that attends to an input image using queries whose positional embedding is supplied. Args: d

A transformer block with four layers: (1) self-attention of sparse inputs, (2) cross attention of sparse inputs to dense inputs, (3)

SAM predicts object masks from an image and input prompts. Arguments: image_encoder (ImageEncoderViT): The backbone used t

The MPT configuration class. Args: d_model (int): The size of the embedding dimension of the model. n_heads (int): Th

Used by HuggingFace generate when using beam search with kv-caching. See https://github.com/huggingface/transformers/blob/3ec7a47664ebe40c40f

Attempts to add bidirectional_mask to batch if missing. Raises: KeyError if bidirectional_mask is missing and can't be inferred

Expects a torch tensor with shape Bx4. Requires the original image size in (H, W) format.

Expects batched images with shape BxCxHxW and float format. This transformation may not exactly match apply_image. apply_image is

Converts a HuggingFace Causal LM to a Prefix LM. Supported HuggingFace model classes: - `GPT2LMHeadModel` - `GPTNeoForCausalLM`