The Low-Rank Adaptation (LoRA) architecture proposed by Hu et al. (2021). See https://arxiv.org/pdf/2106.09685.pdf. LoRA adapts a model by re

Lora Config that applies vector-based random matrix adaptation. It adds trainable matrices 'd' and 'b' while keeping the original LoRA matric

The adapter architecture proposed by Pfeiffer et al. (2020). See https://arxiv.org/pdf/2005.00247.pdf.

The Prefix Tuning architecture proposed by Li & Liang (2021). See https://arxiv.org/pdf/2101.00190.pdf. Args: encoder_prefix (bool):

The 'Infused Adapter by Inhibiting and Amplifying Inner Activations' ((IA)^3) architecture proposed by Liu et al. (2022). See https://arxiv.o

Representation of a Prefix Tuning layer within one Transformer layer. This class implements `AdapterLayerBase` for compatibility with adapter

LoRA Config that applies DoRA, which is a variant of LoRA that decomposes the weight matrix into a magnitude and directional component. S

Lora Config that applies vector-based random matrix adaptation. It adds trainable matrices 'd' and 'b' while keeping the original LoRA matric

The MTL-LoRA architecture, proposed by Yang et al. (2024), combine LoRA with multi-task learning. See https://arxiv.org/pdf/2410.09437.pdf. T

The Prompt Tuning architecture proposed by Lester et al. (2021). See https://arxiv.org/pdf/2104.08691.pdf Args: prompt_length (int):

The UniPELT adapter architecture proposed by Mao et al. (2022). See https://arxiv.org/pdf/2110.07577.pdf.

The adapter architecture proposed by Houlsby et al. (2019). See https://arxiv.org/pdf/1902.00751.pdf.

The Compacter++ architecture proposed by Mahabadi et al. (2021). See https://arxiv.org/pdf/2106.04647.pdf.

Low-Rank Linear Subspace ReFT method proposed in Wu et al. (2024). See https://arxiv.org/pdf/2404.03592.

The parallel adapter architecture proposed by He et al. (2021). See https://arxiv.org/pdf/2110.04366.pdf.

Implementation of various activation function.

Models the input and output states of a bottleneck adapter layer. Args: hidden_states (torch.Tensor): The layer input/ output hidden

The Mix-And-Match adapter architecture proposed by He et al. (2021). See https://arxiv.org/pdf/2110.04366.pdf.

A class providing methods for saving and loading prediction head modules from the file system. Model classes supporting configurable head mo

Represents an adapter setup of a model including active adapters and active heads. This class is intended to be used as a context manager usi

Composes multiple adaptation method configurations into one. This class can be used to define complex adaptation method setups.

Models the input and output states of a LoRA layer. Args: layer_input (torch.Tensor): The input states to the adapted layer. hidd

Defines the main interface for integrating adapter methods into a model class. This interface translates generic accessor names to model-spec

Models the input and output states of a prefix tuning layer. Args: key_states (torch.Tensor): The key states of the attention layer.

The Compacter architecture proposed by Mahabadi et al. (2021). See https://arxiv.org/pdf/2106.04647.pdf.

The adapter architecture proposed by Houlsby et. al. (2019). See https://arxiv.org/pdf/1902.00751.pdf.

Holds context information during a forward pass through a model. This class should be used via the ``ForwardContext.wrap()`` method. Not

The adapter architecture proposed by Pfeiffer et al. (2020). See https://arxiv.org/pdf/2005.00247.pdf.

A class providing methods for saving and loading AdapterFusion modules from the file system.

Holds information about an adapter publicly available on the Hub. Returned by :func:`list_adapters()`. Args: source (str): The s

A class providing methods for saving and loading adapter modules from the Hub, the filesystem or a remote url. Model classes passed to this

Variation of LoReft without orthogonality constraint and projection subtraction as proposed in Wu et al. (2024). See https://arxiv.org/pdf/2404.0

Coupling Block following the GLOW design. The only difference to the RealNVP coupling blocks, is the fact that it uses a single subnetwork to

Coupling Block following the NICE design.

Variation of LoReft without orthogonality constraint.

The model layer that holds all Prefix Tuning prefixes. While each Transformers layer has its own prefix, this layer is shared across all Tran

Prompt Tuning implementation. Args: model_config: The model configuration. adapters_config: The adapter configuration.

Implementation of an AdapterFusion block.

Credit: G. Glavaš & I. Vulić Based on paper "Is Supervised Syntactic Parsing Beneficial for Language Understanding? An Empirical Investigatio

Data collator that will dynamically pad the inputs for multiple choice received. Args: tokenizer ([`PreTrainedTokenizer`] or [`PreTr

Dynamic version of adapter fusion with a value matrix and regularization. See https://arxiv.org/pdf/2005.00247.pdf.

This class manages the setup and configuration of adapter modules in a pre-trained model.

This class is adapted from the compacter implementation at https://github.com/rabeehk/compacter

based on allennlp.training.metrics.AttachmentScores Computes labeled and unlabeled attachment scores for a dependency parse. Note that the in

Generate a Prompt and concatenate it with the input. This is the training time version of prompting a model. Calling the injected `prompt` module

Static version of adapter fusion without a value matrix. See https://arxiv.org/pdf/2005.00247.pdf.

A class providing helper methods for saving and loading module weights.

Mimics the inner EAT backbone (model.local_encoder, model.blocks).

Base class for adapter tests. Defines basic functions and attributes with default values which are used in the tests. Model test classes should in

Implementation of a sequential bottleneck adapter block.

The subset of arguments related to adapter training. Args: train_adapter (bool): Whether to train an adapter instead of the full mod

Base class for all adaptation methods. This class does not define specific configuration keys, but only provides some common helper methods.

Base class that models the architecture of an adapter fusion layer.