Test module with different parameter shapes for testing parametrization.

This class is the base module for the [LLM.int8()](https://arxiv.org/abs/2208.07339) algorithm. To read more about it, have a look at the pap

container for quantization state components to work with Params4bit and similar classes

A parametrization module that handles dequantization of a 4-bit quantized parameter. The parameter data is expected to be already quantized

Mock library handler that defers errors until native methods are called. This class serves as a fallback when the native bitsandbytes librar

Minimal model with a frozen 4-bit base layer and a trainable adapter.

XPU native library with SYCL USM paged memory support.

Embedding class to store and retrieve word embeddings from their indices.

This is the base class similar to Linear4bit. It implements the 4-bit quantization algorithm presented in [QLoRA](https://arxiv.org/abs/2305.

This class implements [LLM.int8()](https://arxiv.org/abs/2208.07339) algorithm for embedding layer Quantization API is similar to Linear8bit

A global optimizer manager for enabling custom optimizer configs.

Per-thread LUT cache with multiple slots to avoid rebuilding when alternating codebooks

This class is the base module for the 4-bit quantization algorithm presented in [QLoRA](https://arxiv.org/abs/2305.14314). QLoRA 4-bit linear

Implements the FP4 data type.

Implements the NF4 data type. Constructs a quantization data type where each bin has equal area under a standard normal distribution N(0, 1) that

Custom embedding layer designed to improve stability during training for NLP tasks by using 32-bit optimizer states. It is designed to reduce gra

Regression tests for #1342 and #1690: quantization must handle non-contiguous tensors correctly.

The cache-gate hooks must stay balanced under activation checkpointing. ``use_reentrant=False`` checkpointing aborts its backward recompute mid-f

Reference: https://hf.co/papers/2409.03137