Create a dictionary of AQUtil instances for each quantized layer; Run forward pass on each sample in inps_tensor; write output activations to

Parallel version of init_aq_engines; works on lists of input/output tensors

Create initial codes and codebooks using residual K-means clustering of weights :params reference_weight, num_codebooks, out_group_size, in_g

Iterate parameters and optimizer states; skip parameters that do not require grad

Load quantized model by dequantizing it

Load quantized model

load student model for fine-tuning. If dequantize is set, dequantize all quantized weights to accumulate full grads

Load unquantized model with frozen parameters

Load model, optimizer state dict and metadata saved via save_training_state; update parameters in-place

Execute function normally or with checkpointing, depending on checkpoint_enabled. Forward **checkpoint_kwargs

Return first element if input is Sequence, otherwise return input

In distributed setting, let only group_size processes enter at a time :param local: if True, the limit is enforced within each host, i.e. dis

create a QuantizedLinear with specified args based on the collected hessian (XTX) data

main entry point to functions for model quantization

Save model for either P- or PV-Tuning using the appropriate saver

Save consolidated model state dict from P-only tuning, can be exported via convert_legacy_model_format.py

Save consolidated model from PV tuning, can be exported later via convert_legacy_model_format.py

Save dequantized model state in the same format as returned by AQLM calibration (main.py)

Split examples that exceed split_max_length into multiple sub-examples

Split all quantized weights between ranks in a distributed setup; uses greedy knapsack heuristic. Note that unlike FSDP, this heuristic will

Trigger FullyShardedDataParallel lazy init in the correct order to allow both training and eval

Undo the effect of wrap_codes_for_fsdp_; modifies state dict in-place

Test that the dequantized model parameters still match the dequantized_to_master dictionary

Make this module compatible with FullyShardedDataParallel; modifies state dict in-place

Utility for parallelism: replace the specified parameters of self.quantized_weight, then run beam search

Utility for parallelism: replace the specified parameters of self.quantized_weight, then compute MSE

Update codes using beam search to minimize L2 error in code values (regardless of activations) :param reference_weight: a target for L2 error

:param XTX: pairwise products of input features matmul(X.transpose(), X), shape: [in_features, in_features] :note: if XTX is divided by datas