Builds a model instance for testing based on the provied model configuration.

Sets the adapter modules to be used by default in every forward pass. If no adapter with the given name is found, no module of the re

Gets the config dictionary for a given adapter. Args: adapter_name (str): The name of the adapter. Returns:

Replicates a given list of tensors based on the shape of the reference tensor (first argument).

Loads the module weights from the given directory. Override this method for additional loading actions. If adding the loaded weights

Generates a set of method test classes for a given model test base. Args: model_test_base (type): The base class for the model tests

Iterates over all layers of the model. This abstract method has to ne implemented by every implementing model.

Matches the shapes of query, key and value matrices for parallel composition.

Forward pass through the adapter layer. NOTE: This method should only be called if the calling module directly inherits from BottleneckLayer.

Loads a pre-trained pytorch adapter module from the local file system or a remote location. Args: adapter_name_or_path (

The forward pass through a prediction head configuration. There are three ways to specify the used prediction head configuration (in

Adds AdapterFusion to the model with alll the necessary configurations and weight initializations Args: adapter_names (F

Sets the model into mode for training the given adapters.

separate heads

Tries to match the given criteria to an existing adapter. Return the adapter config if a match is found, otherwise None.

Saves an adapter and its configuration file to a directory so that it can be shared or reloaded using `load_adapter()`. Args

In-place version of adjust_tensors_for_parallel().

Adds a prefix to an attention mask. The length of the prefix is determined by the `prefix_attention_mask_length` attribute in the ForwardCont

Applies a function to all adapter layers of the model.

Override load_state_dict to handle key remapping between static and flex models. Static models (e.g., BartForConditionalGeneration,

Adds a sequence classification head on top of the model. Args: head_name (str): The name of the head. num_la

Loads a model prediction head from a directory where it was saved using `save_head()`. Args: save_directory (str): Path to the di

Parses and validates a setup of adapters. Args: adapter_composition: The adapter setup to be parsed. level (int, optional):

Saves a model prediction head to a directory such that it can be reloaded using `load_head()`. Args: save_directory (str): Path t

Run evaluation and return metrics. The calling script will be responsible for providing a method to compute metrics, as they are

Helper function to fix the torch seed on cpu and gpu for initializing adapters with the same weights. Is only executed if the config provides

Calculates the hash of a given adapter configuration which is used to identify this configuration. Returns: str: The resulting hash

Setup model for adapter training based on given adapter arguments. Args: model (_type_): The model instance to be trained. adapte

Applies a function to all direct childs of the model if they are a instance of AdapterLayerBase.

Add a new embedding to the model. If a reference embedding and reference tokenizer are provided tokens in the present in both tokeniz

Loads a pre-trained AdapterFusion layer from the local file system. Args: adapter_fusion_name_or_path (str):

Checks if the model supports a given adapter type. Args: adapter_type (str): The adapter type to check. Returns

Check wether the adapter name is present in the model's adapter config and has been created.

Retrieves the correct composition function based on the mapping in 'composition_to_func_map'.

Returns the adapter module with the given name. Args: adapter_name (str): The name of the adapter module.

Saves the module config and weights into the given directory. Override this method for additional saving actions. Args:

Saves an AdapterFusion layer and its configuration file to a directory so that it can be shared or reloaded using `load_adapter_fusio

Sets the model into mode for training of adapter fusion determined by a list of adapter names.

Checks whether the given composition is valid. Args: parent (AdapterCompositionBlock): The parent composition block.

Performs the composition operation between existing and injected weights.

Gets the config dictionary for a given AdapterFusion. Args: fusion_name (Union[str, List[str]]): The name of the Adapter