Create a AutoAugment transform :param config_str: String defining configuration of auto augmentation. Consists of multiple sections separate

Returns the names of available CLIP models

Create a 1D, 2D, or 3D average pooling module.

Crop and resize bounding boxes Args: bboxes: Bounding boxes to crop and resize crop_coords: Coordinates of the crop (

Builds the WebDataset FourM pre-training dataloader based on the given arguments. Args: data_path: Path to the dataset. all_d

Concatenate decoder tensors from different modalities. Args: mod_dict (dict): A dictionary containing information for eac

Convert bounding boxes to a string. xmin, ymin, xmax, ymax are mapped to v0, v1, v2, v3 special tokens. Args: bboxes: Bo

Convert instances dict to list of lists where each list takes the form: [xmin, ymin, xmax, ymax, class_name, score]

Convert instances dict to list of lists where each list takes the form: [human, xmin xmax ymin ymax global val1 val2 ... val10 pose val1 val2

Convert humanposes to a string range of global orientation: [-1, 1] range of object pose: [-1, 1] range of shape (bet

Convert colors to v0= v0= ...

Convert palettes to a string

Converts a string of bboxes to a list of bboxes. Args: bboxes_str (str): String of bboxes bins (int): Number of bins (defaul

Convert the torso of the model to float16.

Convert applicable model parameters to fp16

Crop the sample using crop coordinates.

Decodes a sequence of color palettes from a model dictionary. Args: mod_dict (dict): Model output dictionary. key (str): Key

Decodes human poses tokenized with text + BMLP

Decodes (denormalizes) an RGB image from a model dictionary. Args: mod_dict (dict): Model output dictionary. key (str): Key

Decodes a sequence of metadata tokens into a dictionary of metadata. Args: mod_dict (dict): Model output dictionary. key (st

Decodes quantized latent codes back to an image. Args: quant: Quantized latent code of shape B D_Q H_Q W_Q Returns:

Decodes a sequence of SAM instance tokens into the instance representation. Args: mod_dict (dict): Model output dictionary.

Decodes a sequence of Canny edges tokens from a model dictionary into an RGB image. Args: mod_dict (dict): Model output dictionary.

Decodes a sequence of CLIP tokens from a model dictionary into an PCA representation. Args: mod_dict (dict): Model output dictionary

Decodes a sequence of depth tokens from a model dictionary into an RGB image. Args: mod_dict (dict): Model output dictionary.

Decodes a sequence of DINOv2 spatial tokens from a model dictionary into an PCA representation. Args: mod_dict (dict): Model output

Decodes a sequence of DINOv2 global tokens from a model dictionary. Args: mod_dict (dict): Model output dictionary. key (str

Decodes a sequence of ImageBind spatial tokens from a model dictionary into an PCA representation. Args: mod_dict (dict): Model outp

Decodes a sequence of ImageBind global tokens from a model dictionary. Args: mod_dict (dict): Model output dictionary. key (

Decodes a sequence of surface normal tokens from a model dictionary into an RGB image. Args: mod_dict (dict): Model output dictionar

Decodes a sequence of RGB tokens from a model dictionary into an RGB image using a ControlNet. Args: mod_dict (dict): Model output d

Decodes a sequence of SAM edges from a model dictionary into an RGB image. Args: mod_dict (dict): Model output dictionary. t

Decodes a sequence of semantic segmentation tokens from a model dictionary into an RGB image. Args: rgb_img (torch.Tensor): RGB imag

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). This is the same as the DropConnect impl I created for Ef

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). Implementation from timm: https://github.com/huggingface

Given an instance and a border function `border_fn`, extends the instance points with crossing points between the instance and the crop bord

Forward pass of the encoder, quantizer, and decoder. Args: x: Input image tensor of shape B C H W or B H W in case

Computes the loss based on the specified loss type. Args: y (torch.Tensor): Decoder output. Shape (B, M, D). target_i

Concatenates and mask decoder tensors based on provided modality information. This function consolidates decoder tokens from multiple modalit

Modification of forward_mask_decoder for MaskGIT generation, with support for batching

Modification of forward_mask_decoder for ROAR generation, with support for batching

Concatenates and mask encoder tensors based on provided modality information. This function consolidates encoder tokens from multiple modalit

Computes the modality-wise loss. Args: y (torch.Tensor): Decoder tokens. Shape (B, M, D). target_ids (torch.Tensor):

Run a forward step of the network Args: batch (Dict): Dictionary containing batch data train (bool): Flag ind

Computes the token-wise loss. Args: y (torch.Tensor): Decoder tokens. Shape (B, M, D). target_ids (torch.Tensor): Gro

Converts all `BatchNorm2d` and `SyncBatchNorm` layers of provided module into `FrozenBatchNorm2d`. If `module` is itself an instance of eithe

Returns a single nn.Linear layer with the LoRA matrix fused into the original one.

Extra tokens that are used for bounding box coordinates, xmin, ymin, xmax, ymax, but also other modalities like color maps, metadata, or pose

Parses the arguments from the command line.

Parses the arguments from the command line.

Parses the arguments from the command line.