Add an adapter with the given name and config.

Add task prompt and end prompt to each item in the data

Calculate and return importance scores

Consolidate parameters from current and previous tasks based on importance scores Args: current_weights: Dictionary of current task

r""" Loads a configuration file from a json file. Args: path_json_file (`str`): The path to the json file

r""" This method loads the configuration of your adapter model from a directory. Args: pretrained_model_name_or_path (`st

Identify all LoRA parameters in the model

Returns samples from buffer using nearest neighbour approach

Get the state dict of the Peft model. Args: model ([`PeftModel`]): The Peft model. When using torch.nn.DistributedDataParallel, Deep

Returns the prompt embedding to save when saving the model. Only applicable when `peft_config.peft_type != PeftType.LORA`.

Extract task name from file path

Return True if module is trainable under adaption prompt fine-tuning.

Apply rotary position embedding to query states in the Llama model. This function was adapted from: https://github.com/huggingface/trans

Rotate half the hidden dims of the input. This function was duplicated verbatim from: https://github.com/huggingface/transformers/blob/1

Prepare the config based on the llama model type.

Solves the GEM dual QP described in the paper given a proposed gradient "gradient", and a memory of task gradients "memories"

Add the examples as key-value pairs to the memory dictionary with content,attention_mask,label tuple as value and key determined by k

Randomly sample n items from data. If data has fewer items than n, return all data