enumerate available model architectures based on config files

Returns the names of available CLIP models

return all models having the specified pretrain tag

Load annotation from directory. Args: img_dir (str): Path to image directory img_suffix (str): Suffix of images.

Load checkpoint from a file or URI. Args: model (Module): Module to load checkpoint. filename (str): Accept local filepath, URL,

load checkpoint through URL scheme path. Args: filename (str): checkpoint file name with given prefix map_location (s

Example >>> dataset = load_dataset("coco_caption", cfg=None) >>> splits = dataset.keys() >>> print([len(dataset[split]) for split in

In distributed setting, this function only download checkpoint at local rank 0.

Load model and its related preprocessors. List all available models and types in registry: >>> from lavis.models import model_zoo >>

Load a CLIP model Parameters ---------- name : str A model name listed by `clip.available_models()`, or the path to a model checkp

In distributed setting, this function only download checkpoint at local rank 0.

Load preprocessor configs and construct preprocessors. If no preprocessor is specified, return BaseProcessor, which does not do any preproce

Load state_dict to a module. This method is modified from :meth:`torch.nn.Module.load_state_dict`. Default value for ``strict`` is set to ``F

Compute segmentation loss.

Compute segmentation loss.

Compute ``pam_cam``, ``pam``, ``cam`` loss.

Build a color wheel. Args: bins(list or tuple, optional): Specify the number of bins for each color range, corresponding to s

Stack InvertedResidual blocks to build a layer for MobileNetV2. Args: out_channels (int): out_channels of block. num_

Pack all blocks in a stage into a ``ResLayer``.

make plugins for ResNet 'stage_idx'th stage . Currently we support to insert 'context_block', 'empirical_attention_block', 'nonlocal_

Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original.

Merge datasets in download_path to storage_path

feature module name accessor

Move files from download_path to storage_path

Move files from download_path to storage_path

Move files from download_path to storage_path

CPU version of multi-scale deformable attention. Args: value (Tensor): The value has shape (bs, num_keys, mum_heads, embed_di

nn.Module: the normalization layer named "norm0"

nn.Module: the normalization layer named "norm1"

nn.Module: normalization layer after the first convolution layer

nn.Module: the normalization layer named "norm1"

nn.Module: normalization layer after the second convolution layer

nn.Module: normalization layer after the second convolution layer

Normalize input grid from [-1, 1] to [0, 1] Args: grid (Tensor): The grid to be normalize, range [-1, 1]. Returns: Tensor: Nor

Convert a numpy image or a batch of images to a PIL image.

PAM feature classification.

Parse the prefix of a uri. Args: uri (str | Path): Uri to be parsed that contains the file prefix. Examples:

Patch the forward method of a module. Args: func (callable): The original forward method. src_type (torch.dtype): Type of input a

Recursively convert normalization layers from FP16 to FP32. Args: module (nn.Module): The modules to be converted in FP16. Returns:

pc: NxC, return NxC

Get testing data after pipeline. Args: idx (int): Index of data. Returns: dict: Testing data after pipeline

Get training data and annotations after pipeline. Args: idx (int): Index of data. Returns: dict: Training da

Print a model with FLOPs for each layer. Args: model (nn.Module): The model to be printed. total_flops (float): Total FLOPs of th

projects hidden states correctly to key/query states

Forward function of PSP module.

Save data to a given ``filepath``. Args: obj (bytes): Data to be saved. filepath (str or Path): Path to write data.

Quantize an array of (-inf, inf) to [0, levels-1]. Args: arr (ndarray): Input array. min_val (scalar): Minimum value to be clippe

Quantize flow to [0, 255]. After this step, the size of flow will be much smaller, and can be dumped as jpeg images. Args: flow

Randomly select an img_scale from given candidates. Args: img_scales (list[tuple]): Images scales for selection. Returns

Generate the first token of answers using decoder and select ${num_ans_candidates} most probable ones. Then select answers from answe

Reduce loss as specified. Args: loss (Tensor): Elementwise loss tensor. reduction (str): Options are "none", "mean" and "sum".

Reduce mean when distributed training.

Register a hook from its cfg. Args: hook_cfg (dict): Hook config. It should have at least keys 'type' and 'priority