The implementation is identical to `lit_gpt.model.Block` with the exception that we replace the attention layer where adaption is implemented.

The implementation is identical to `lit_gpt.model.Block` with the exception that we replace the attention layer where adaption is implemented.

A modification of `lit_gpt.model.CausalSelfAttention` that adds the attention over the adaption prompt.

A modification of `lit_gpt.adapter.CausalSelfAttention` that uses the Adapter V2 Linear class

Args: r: rank of the weight update matrices. To make sense of using LoRA the rank should be smaller than the rank of the weig

The implementation is identical to `lit_gpt.model.GPT` with the exception that the `Block` saves the layer index and passes it down to the attenti

Root Mean Square Layer Normalization. Derived from https://github.com/bzhangGo/rmsnorm/blob/master/rmsnorm_torch.py. BSD 3-Clause License: ht