A class for quantizing the activations. We only support (both sym. and asym.) per-token quantization for the activations.

From GPTQ Repo

This class is a wrapper for the activation quantization. We extract the FP features in the forward pass and quantize the rest using the s

This class is a wrapper for the activation quantization. We extract the FP features in the forward pass and quantize the rest using

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig

A class for quantizing the activations. We only support (both sym. and asym.) per-token quantization for the activations.

A class for quantizing the activations. We only support (both sym. and asym.) per-token quantization for the activations.

r"""This optimizer updates variables with two different routines based on the boolean variable 'stiefel'. If stiefel is True, the var

Mixture-of-Experts (MoE) module. Attributes: dim (int): Dimensionality of input features. n_routed_experts (int): Total numb

The unnormalized Hadamard transform (i.e. without dividing by sqrt(2))

Multi-headed attention from 'Attention Is All You Need' paper

Multi-headed attention from 'Attention Is All You Need' paper

LlamaRotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla

LlamaRotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla

Llama flash attention module. This module inherits from `LlamaAttention` as the weights of the module stays untouched. The only required chan

Llama flash attention module. This module inherits from `LlamaAttention` as the weights of the module stays untouched. The only required chan

LlamaRotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev

LlamaRotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev

Llama attention module using torch.nn.functional.scaled_dot_product_attention. This module inherits from `LlamaAttention` as the weights of t

Llama attention module using torch.nn.functional.scaled_dot_product_attention. This module inherits from `LlamaAttention` as the weights of t

This class implements the Root Mean Square Normalization (RMSN) layer. We use the implementation from LLAMARMSNorm here: https://github.c

From GPTQ Repo

From GPTQ Repo