Set the module into train or eval mode if `mode` is True of False respectively. For train mode (train(True)) if weights are merged we need to

Generates a standardized message to prompt the model with an instruction, optional input and a 'response' field.

Takes a conditioning sequence (prompt) as input and continues to generate as many tokens as requested. The implementation of this function is mod

Returns the LLaMA model name from the checkpoint. Checks the width of the lm_head.weight matrix, as these uniquely identify the model.

Apply context manager under which you can instantiate the model with LoRA. In a nutshell the code inside this function forces to use LoRA variant

Handles boilerplate logic for retrieving and saving the state_dict. This will be upstreamed to Fabric soon.

Store LoRA specific attributes in a class. Args: r: rank of the weight update matrices. To make sense of using LoRA the rank shou

Returns the model state dict with only the adapter weights for saving.

Returns the model state dict with only the adapter weights for saving.

Return state_dict with weights of LoRA's A and B matrices and with biases depending on the `bias` value. Args: model: model with LoRA lay

Freeze all modules except LoRA's and depending on 'bias' value unfreezes bias weights. Args: model: model with LoRA layers bias:

Processes a single sample. This function processes the line to produce the tokenized version of it.

Processes a single sample. Each sample in the dataset consists of: - instruction: A string describing the task - input: A string hold

Processes a single sample. Each sample in the dataset consists of: - instruction: A string describing the task - input: A string hold

Properly pad weight updates with zeros. If, based on `self.enable_lora`, we want to fine-tune queries and values, but not keys, then

Enhanced Transformer with Rotary Position Embedding. Derived from: https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/

Downloads the raw json data file and saves it in the given destination.

Downloads the raw json data file and saves it in the given destination.

Generates a standardized message to prompt the model with an instruction, optional input and a 'response' field.

Return a list of indices so that each slice of `batch_size` consecutive indices correspond to elements of similar lengths. To do this, the in

This is the classic post-training quantization of all linear layers. We quantize in order, i.e. when observing the inputs, we use the outputs