Tokenizer for LLaMA.

Root Mean Square Layer Normalization. Derived from https://github.com/bzhangGo/rmsnorm/blob/master/rmsnorm_torch.py. BSD 3-Clause License: ht

r""" Sampler that samples indices in a way that groups together features of the dataset of roughly the same length while keeping a bit of rand

The implementation is identical to `lit_llama.model.LLaMA` with the exception that the `Block` saves the layer index and passes it down to the att

The implementation is identical to `lit_llama.model.Block` with the exception that we replace the attention layer where adaption is implemented.

A modification of `lit_llama.model.CausalSelfAttention` that adds the attention over the adaption prompt.

Wraps `bnb.nn.Linear8bitLt` and enables instantiation directly on the device and re-quantizaton when loading the state dict. This sh