Cancel out PyTorch rescaling by 1/(1-p)

Track a series of values and provide access to smoothed values over a window or the global series average.

Subclass torch's LayerNorm to handle fp16.

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between

CfgNode represents an internal node in the configuration tree. It's a simple dict-like container that allows for attribute-based access to ke

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Image to Patch Embedding

Construct the embeddings from word and position embeddings.

Construct the embeddings from word and position embeddings.

This is a preliminary implementation to centralize and validate the configuration. May be altered in the future. A helper class to valid

3D Sparse UNet for Distillation.

Gradually warm-up(increasing) learning rate in optimizer. Proposed in 'Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour'. Args:

A wrapper to convert DataLoader as an infinite iterator. Modified from: https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/i

Subclass torch's LayerNorm to handle fp16.

MLP as used in Vision Transformer, MLP-Mixer and related networks

A utility class to create string representation of available model architectures and types. >>> from lavis.models import model_zoo >>> #

A simple wrapper for iterating over multiple iterators. Args: loaders (List[Loader]): List of Iterator loaders. ratios (List

Multi-headed attention from 'Attention Is All You Need' paper

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`OPTDecoderLayer`] Args: config: OPTConfig

This module learns positional embeddings up to a fixed maximum size.

r"""Pointnet set abstrction layer with multiscale grouping Parameters ---------- npoint : int Number of features radii : list

Modified from https://github.com/ChenRocks/UNITER. overlap compute and cuda data transfer (copied and then modified from nvidia apex)

A runner class to train and evaluate a model given a task and datasets. The runner uses pytorch distributed data parallel by default. Future