Check if the given quant_type, dtype and quant_storage are supported on HPU.

Performs a single optimization step. Args: closure (callable, optional): A closure that reevaluates the model and

Return optimizer state, wrapping quantization tensors for FSDP compatibility. FSDP's full_optim_state_dict gathers all tensor states across r

Args: num_embeddings (`int`): The number of unique embeddings (vocabulary size). embedding_dim (`int`

Gets the memory address of the first element of a tenso Args: A (`Optional[Tensor]`): A PyTorch tensor. Returns: `Optional[c

returns dict of tensors and strings to use in serialization via _save_to_state_dict() param: packed -- returns dict[str, torch.Tensor

Replace a module parameter with a 4-bit quantized version using parametrization. This function quantizes an existing parameter in a PyTorch

Get the path to the best matching CUDA/ROCm BNB native library for the given specs. When no override is set, selects from packaged libraries

Load an optimizer state. Arguments: state_dict (`dict`): An optimizer state (should be returned from a call to `s

Base Lion optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. l

Base AdamW optimizer. Arguments: params (`torch.Tensor`): The input parameters to optimize.

Base Adam optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. l

unpacks components of state_dict into QuantState where necessary, convert into strings, torch.dtype, ints, etc. qs_dict: bas

Verifies that the input tensors are all on the same device. An input tensor may also be marked as `paged`, in which case the device placement is

Base LARS optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. l

packed_weight: [N, K/2] uint8, output of `_convert_weight_packed_for_cpu` (final_qweight) quant_state: QuantState that was modified by `_co

Create a bnb optimizer by name.

Dequantize a tensor in blocks of values. The input tensor is dequantized by dividing it into blocks of `blocksize` values. The the absolute m

Extract error types, library names, and technical terms. These are specific, discriminative terms — not general words like 'cuda' which appea

One-line summary of an issue.

Get CUDA/HIP version as a tuple of (major, minor).

Load and tokenize a small subset of Alpaca.

Reference implementation

Base LAMB optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. l

Base SGD optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr

Base Adagrad optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize.

Base RMSprop optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize.

Return a {(major, minor): Path} mapping for all packaged CUDA/ROCm library files.

Split a compact CUDA/ROCm version string from a library filename into (major, minor). CUDA: major is always 2 digits (11, 12, 13...), e.g. '118'

Force cleanup of GPU memory.

Creates the dynamic quantiztion map. The dynamic data type is made up of a dynamic exponent and fraction. As the exponent increase from

Create a LLaMA model from config (no download needed).

Dequantizes a packed 4-bit quantized tensor. The input tensor is dequantized by dividing it into blocks of `blocksize` values. The absolute m

Find issues related to target. Returns list of (score, issue, sig_overlap, token_overlap).

Format a single related-issue result.

Return the torch accelerator module (torch.cuda / torch.xpu).

Get CUDA/HIP specifications.

Try calling native lib.has_avx512bf16_cpu(). Return False explicitly if symbol missing or call fails.

NEON-optimized BF16 to float conversion (4 values at a time)

NEON-optimized float to BF16 conversion (4 values at a time, with rounding)