Creates the binary mask of each instance in the sample.

Creates and returns a function `fn` using crop region coordinates given in crop_points. `fn` receives two input points x and xn and returns a

Extracts the class and file name from a path.

Creates and returns feature extractor from arguments.

Load a pretrained image tokenizer from a checkpoint. Args: tokenizer_id (str): ID of the tokenizer to load (name of the checkpoint f

Compute rectangle enclosing keypoints above the threshold. Args: keypoints (np.array): Keypoint array of shape (N, 3). thresh

Creates and returns model from arguments

Creates and returns model from arguments.

Creates and returns model from arguments.

Creates and returns model from arguments

Creates and returns model from arguments

Layers go from 0 to (num_enc + num_dec + 1) where 0 is the encoder embedding and (num_enc + num_dec + 1) is the projection following the decoder

Creates and returns perceptual loss from arguments.

Returns the PIL resampling mode for the given resample mode string. Args: resample_mode: Resampling mode string

gopen from webdataset, but with s3 support

Crop and resize human poses (and their bounding boxes)

Applies input and target masking to an image tensor Args: tensor: Image tensor num_tokens: Number of tokens in the te

Initialize parts of embedding module that are dependent on dimension of tokens. Should be called when setting up FourM. Args

Initialize parts of module that are dependent on dimension of tokens. Should be called when setting up FourM. Args:

Initialize parts of embedding module that are dependent on dimension of tokens. Should be called when setting up FourM. Args

Initialize parts of embedding module that are dependent on dimension of tokens. Should be called when setting up FourM. Args

Initialize parts of module that are dependent on dimension of tokens. Should be called when setting up FourM. Args:

Weight initialization following MAE's initialization scheme

Weight initialization following MAE's initialization scheme

Weight initialization following MAE's initialization scheme

Applies masking for an image given as input Args: tensor: Image tensor num_tokens: Number of tokens in the tensor

Applies masking for a sequence given as input Args: sequence_str: Sequence string max_tokens: Maximum number of token

Sample a token budget for the input Args: num_input_tokens: Number of tokens in the input Returns: Token bud

Calculates iou of the input bounding boxes

Check if a model exists within a subset of modules Args: model_name (str) - name of model to check module_names (tuple, list, set)

Temperature that decays the temperature inversely proportional to the token schedule.

Main function for training and evaluation.

Main function for training and evaluation.

Main function for training and evaluation.

Converts v0, v1, v2, v3 textual representation into int. When tokens=True, inp is mapped to its corresponding token id.

Optionally mask out invalid regions and concat mask to images. Useful when tokenizing simulated data that contains unlabeled regions. Args:

Copy the master parameter data back into the model parameters.

Fetch a model entrypoint for specified model name

Copy the gradients from the model parameters into the master parameters from make_master_params().

Abitrary temperature schedule for one over x

Preprocesses the input image tensor before feeding it to the encoder. If self.undo_std, the input is first denormalized from the ImageNet

Convert quaternion representation to rotation matrix. Args: quat (torch.Tensor) of shape (B, 4); 4 <===> (w, x, y, z). Returns:

Create a RandAugment transform :param config_str: String defining configuration of random augmentation. Consists of multiple sections separa