Stores the configuration of an [`AdaptionPromptModel`].

This is the configuration class to store the configuration of a [`LoraModel`]. Args: r (`int`): Lora attention dimension. ta

Base class for the base and weights of the conditional layer norms.

Base class for the base and weights of the conditional layer norms.

A class that keeps all conversation history.

Implements Adapter controller module which controls the logics of putting adapter layers within transformer's layers.

This module generates the weights for all the meta adapter layers given the task embeddings and layer id.

This module generates the weights for all the meta adapter layers given the task embeddings and layer id.

This module generates the weight and bias for the task conditioned layer norm.

This module generates the weight and bias for the task conditioned layer norm.

Base class for each adapter weights

Base class for each adapter weights

Subclass torch's LayerNorm to handle fp16.

Subclass torch's LayerNorm to handle fp16.

Implements Meta Adapter controller module, in which the adapter layers' weights are generated from a unique hyper-network.

Implements Adapter controller module which controls the logics of putting adapter layers within transformer's layers.

Processes an answer similar to Eval AI copied from https://github.com/facebookresearch/mmf/blob/c46b3b3391275b4181567db80943473a8

Main module controlling task embeddings.

Conventional Adapter layer, in which the weights of up and down sampler modules are parameters and are optimized.

Conventional Adapter layer, in which the weights of up and down sampler modules are parameters and are optimized.

Conventional Adapter layer, in which the weights of up and down sampler modules are parameters and are optimized.

This modules contains the hyper-nets for the feed forward and self-attention modules and it generates the adapter's weights and layer norm's w

This modules contains the hyper-nets for the feed forward and self-attention modules and it generates the adapter's weights and layer norm's w

Multi-headed attention from 'Attention Is All You Need' paper

Transformer decoder consisting of *config.decoder_layers* layers. Each layer is a :class:`BartDecoderLayer` Args: config: BartConfig

Implements BERT version of Adam algorithm with weight decay fix. Params: lr: learning rate warmup: portion of t_total for the warm

Implementation of the gelu activation function. For information: OpenAI GPT's gelu is slightly different (and gives slightly different results

Wraps another sampler to yield a mini-batch of indices. It enforces that the batch only contain elements from the same group. It also tri

Bottleneck adapter in a dense layer. The adapter can be applied after the multi-head attention layer and/or after the feed-forward layer.

Multi-headed attention from 'Attention Is All You Need' paper

Parameter-Efficient Fine-Tuning Model. Base model encompassing various Peft methods. Args: model ([`PreTrainedModel`]): The base tra

This module generates the task-embeddings from the initial feeded task embeddings.

This module generates the task-embeddings from the initial feeded task embeddings.

Implements Adapter controller module which controls the logics of putting adapter layers within transformer's layers.

Base class for adapter layer's outputs.

Base class for adapter layer's outputs.

For a set of image sizes and feature maps, computes a set of anchors.

This module learns positional embeddings up to a fixed maximum size.

This R-CNN transformation scales the box's width and height by exp(dw), exp(dh) and shifts a box's center by the offset (dx * width, dy *

Hypercomplex Adapter layer, in which the weights of up and down sampler modules are parameters are 1/n times of the conventional adapter layers, w

Hypercomplex Adapter layer, in which the weights of up and down sampler modules are parameters are 1/n times of the conventional adapter layers, w

Structure that holds a list of images (of possibly varying sizes) as a single tensor. This works by padding the images to the same size,

Structure that holds a list of images (of possibly varying sizes) as a single tensor. This works by padding the images to the same size,

A single training/test example for the language model.

LXRT Model.

This is the low-rank adapter, in which each adapter is composed of two rank-one matrices.

This is the low-rank adapter, in which each adapter is composed of two rank-one matrices.

A ResNet class that is similar to torchvision's but contains the following changes: - There are now 3 "stem" convolutions as opposed to 1, wi