Resize images, extract features, etc. Args: x: Input dict. domain: Domain we are training on. feature_extractor: Optiona

r"""Fills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distributi

Visualizes bounding boxes on the image. Args: img (np.array): Image to draw bounding boxes on. bboxes_str (str): String cont

Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [

Sine-cosine positional embeddings from MoCo-v3, adapted back to 1d Returns positional embedding of shape (1, N, D)

Sine-cosine positional embeddings as used in MoCo-v3 Returns positional embedding of shape (1, N, D) where N = W*H

Return raymond light nodes for the scene.

Decodes quantized latent codes back to an image. Args: quant: Quantized latent code of shape B D_Q H_Q W_Q Returns:

Log several reconstructed images to wandb. Args: model: Model to evaluate. data_loader: Validation set data loader. devic

Log several reconstructed images to wandb. Args: model: Model to evaluate. feature_extractor: Optional feature extractor.

Compute validation image metrics (FID, LPIPS, Inception, MS-SSIM, PSNR, MSE) using torchmetrics and compute codebook usage stats. Args:

Compute validation image metrics (FID, LPIPS, Inception, MS-SSIM, PSNR, MSE) using torchmetrics and compute codebook usage stats. Args:

Perform one evaluation epoch and return stats. As during training, the resolution is randomly sampled from train_res_choices. Args:

Perform one evaluation epoch and return stats. As during training, the resolution is randomly sampled from train_res_choices. Args:

Horizontal flipping the instances in a sample.

Create a linear module.

Removes the spines from the given axis. Args: ax (matplotlib.axes.Axes): Axis to remove spines from

Sample a value from a uniform distribution between min_val and max_val Args: min_val: Minimum value max_val: Maximum value R

Find all modules of a certain class (or union of classes) that are direct or indirect descendants of other modules of a certain class (or uni

Freeze parameters.

Freeze or unfreeze parameters of the specified modules and those of all their hierarchical descendants. This is done in place. Args:

Creates a conditional diffusion pipeline with the given scheduler. Args: scheduler: Scheduler to use for the diffusion pipeline.

Sine-cosine positional embeddings as used in MoCo-v3 Returns positional embedding of shape [B, H, W, D]

Sine-cosine positional embeddings as used in MoCo-v3

Builds the FourM pre-training dataset based on the given arguments. This function should mainly used for smaller datasets (e.g. validation sets),

Constructs an MLP model from a model ID string, see "Scaling MLPs: A Tale of Inductive Bias" (https://arxiv.org/abs/2306.13575). Args:

Returns list of utf-8 byte and a corresponding list of unicode strings. The reversible bpe codes work on unicode strings. This means you

Concatenate encoder tensors from different modalities. Args: mod_dict (dict): A dictionary containing information for each modali

Evaluate a function without caching intermediate activations, allowing for reduced memory at the expense of extra compute in the backward pas

Computes the codebook usage for a given set of encoded tokens, by computing the percentage of unique tokens in windows of a given size. The windo

Reconstruction loss computation. Args: model_output: Model predictions. images: Target. loss_fn: Loss function to use. Ca

A counter for the `thop` package to count the operations in an attention operation. Meant to be used like: macs, params = thop.pr

Create a model Args: model_name (str): name of model to instantiate pretrained (bool): load pretrained ImageNet-1k weights if tru

Decodes a sequence of RGB tokens from a model dictionary into an RGB image. Args: mod_dict (dict): Model output dictionary.

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). This is the same as the DropConnect impl I created for Ef

Scales the noise schedule betas so that last time step has zero SNR. See https://arxiv.org/abs/2305.08891 Args: betas: the initial di

Gets bounding box of the given instance

Returns the crop heights and widths from the crop coordinates.

Return set of symbol pairs in a word. Word is represented as tuple of symbols (symbols being variable-length strings).

Get parameters for ``crop`` for a random sized crop. Args: img (PIL Image): Image to be cropped. scale (tuple): range