Functions — github.com/IST-DASLab/MoE-Quant

hub / github.com/IST-DASLab/MoE-Quant / functions

Functions65 in github.com/IST-DASLab/MoE-Quant

⨍ Functions 65 ◇ Types & classes 3

↓ 5 callersMethodupdate

Update the estimate of Hessian matrix from a batch of data. Args: input: batch of layer inputs

src/gptq.py:74

↓ 3 callersFunctionis_dist_available_and_initialized

()

src/dist_utils.py:20

↓ 3 callersFunctionround_fp

(x: torch.Tensor, dtype: torch.dtype = None)

src/quant_utils.py:57

↓ 3 callersFunctiontl_round_fp

(x, dtype)

src/quant_utils.py:38

↓ 2 callersFunctiondequantize

(qx: torch.Tensor, scale: torch.Tensor, qzero: torch.Tensor, dtype: torch.dtype = None)

src/quant_utils.py:65

↓ 2 callersMethodhas_hessian_issues

(self)

src/gptq.py:69

↓ 2 callersFunctionis_main

()

src/dist_utils.py:36

↓ 2 callersMethodquantize

(self, bits: int | float)

src/gptq.py:216

↓ 2 callersMethodreset

(self)

src/gptq.py:111

↓ 2 callersFunctiontl_dequantize

(qx, scale, qzero, dtype)

src/quant_utils.py:48

↓ 2 callersFunctiontl_quantize

(x, scale, qzero, maxq)

src/quant_utils.py:43

↓ 1 callersMethod_get_hessian_inverse

(self)

src/gptq.py:221

↓ 1 callersMethod_quantize

Quantize the weight matrix using GPTQ

src/gptq.py:157

↓ 1 callersMethod_validate_layer

(layer)

src/gptq.py:66

↓ 1 callersFunctionaddvv_triton

( vec_a: torch.Tensor, vec_b: torch.Tensor, mat_c: torch.Tensor, )

src/gptq_loop.py:91

↓ 1 callersFunctiondequantize_weight_from_fp8

(W, s)

src/quant_utils.py:267

↓ 1 callersFunctionfind_quantization_meta

Find quantization metadata over dim=-1 x: (..., C), weight bit_width: int symmetric: bool, whether to set qzero to the middle dty

src/quant_utils.py:75

↓ 1 callersFunctionget_rank

()

src/dist_utils.py:30

↓ 1 callersFunctionget_resume_block_idx

(save_dir: os.PathLike)

quant.py:91

↓ 1 callersFunctionget_world_size

()

src/dist_utils.py:24

↓ 1 callersFunctiongptq_loop_graph

CUDA Graph wrapper for GPTQ loops

src/gptq_loop.py:108

↓ 1 callersFunctionis_subset

(set1: set, set2: set)

pack_quantized_model.py:53

↓ 1 callersFunctionis_subset

(set1: set, set2: set)

quant.py:87

↓ 1 callersFunctionmain

()

pack_quantized_model.py:112

↓ 1 callersFunctionmain

()

quant.py:100

↓ 1 callersFunctionmse_scale

Find the optimal scale for quantization with respect to the MSE loss x: (..., C), weight p: (P), shrinkage factors scale: (...), init

src/quant_utils.py:144

↓ 1 callersFunctionpack_weight

( weight: dict[torch.Tensor], bits: int, sym: bool, group_size: Optional[int] = None, )

pack_quantized_model.py:57

↓ 1 callersFunctionparse_args

()

pack_quantized_model.py:20

↓ 1 callersFunctionparse_args

()

quant.py:26

↓ 1 callersFunctionprepare_fineweb_edu

( tokenizer: AutoTokenizer, max_sequence_length: int, num_calibration_samples: Optional[int] = No

src/data_utils.py:92

↓ 1 callersFunctionprepare_open_platypus

( tokenizer: AutoTokenizer, max_sequence_length: int, num_calibration_samples: Optional[int] = No

src/data_utils.py:61

↓ 1 callersFunctionprepare_open_thoughts

( tokenizer: AutoTokenizer, max_sequence_length: int, num_calibration_samples: Optional[int] = No

src/data_utils.py:17

↓ 1 callersFunctionprepare_quantization_config

(args: argparse.Namespace)

pack_quantized_model.py:78

↓ 1 callersMethodquantization_pre_step

Preparatory step with hessian regularization and weight reshaping.

src/gptq.py:123

↓ 1 callersFunctionquantize

(x: torch.Tensor, scale: torch.Tensor, qzero: torch.Tensor, maxq: torch.Tensor)

src/quant_utils.py:61

↓ 1 callersFunctionquantize_error_triton

( x: torch.Tensor, qx: torch.Tensor, error: torch.Tensor, scale: torch.Tensor, qzero: torc

src/gptq_loop.py:43

↓ 1 callersFunctionsplit_thought_solution

(text: str)

src/data_utils.py:9

↓ 1 callersFunctiontl_dequantize_quantized

(x, scale, qzero, maxq, dtype)

src/quant_utils.py:53

↓ 1 callersFunctiontl_pow

(x, a)

src/quant_utils.py:26

↓ 1 callersFunctiontl_round

(x)

src/quant_utils.py:31

↓ 1 callersFunctionupdate_handle_hook

(name)

quant.py:249

Method__init__

( self, layer: nn.Module, group_size: Optional[int] = None, sym: bool = False,

Functionaddvv_triton_kernel

( vec_a_ptr, vec_b_ptr, mat_c_ptr, size_a: int, size_b: int, BLOCK_SIZE_B: tl.constexp

Functionbroadcast_parameters

(module: nn.Module, src: Any = 0, group: Optional[Any] = None)

src/dist_utils.py:45

Functioncan_dequantize_from_fp8

(state_dict: dict[str, torch.Tensor])

src/quant_utils.py:300

Functiondequantize_linear_weight

( qweight: torch.Tensor, scale: torch.Tensor, zero: torch.Tensor, perm: Optional[torch.Tensor]

src/quant_utils.py:243

Functiondequantize_quantized

( x: torch.Tensor, scale: torch.Tensor, qzero: torch.Tensor, maxq: torch.Tensor, dtype: torch.dtype = None

src/quant_utils.py:69

Functiondequantize_state_dict

(state_dict: dict[str, torch.Tensor], dtype: torch.dtype = torch.float16)

src/quant_utils.py:286

Functiongather_into_tensor

(tensor, dim: int = 0)

src/dist_utils.py:50

Functionget_number_of_rows_and_cols

(layer)

src/model_utils.py:11

Functionget_quantization_grid

Get the quantization grid for the weight matrix weight: (..., (R), C) scale: (..., (R), C) qzero: (..., (R), C) maxq: ()

src/quant_utils.py:196

Functionget_relative_mse_error

(q: torch.Tensor, w: torch.Tensor, H: Optional[torch.Tensor] = None)

src/quant_utils.py:259

Functiongptq_loop

Quantize weight tensor with GPTQ algorithm weight: (C, R), transposed weight tensor to quantize, modified in-place and returned hessian_i

src/gptq_loop.py:193

Functioninv_sym

More efficient and stable inversion of symmetric matrices.

src/linalg_utils.py:8

Functionload_param_shard

(weight_dir: str, weight_path: str)

src/loading_utils.py:6

Functionmse_scale_triton_kernel

( x_ptr, p_ptr, scale_ptr, qzero_ptr, maxq_ptr, dtype_ptr, norm: float, p_size

src/quant_utils.py:105

Functionprepare_calibration_dataset

( dataset_name: str, tokenizer: AutoTokenizer, max_sequence_length: int, num_calibration_sam

src/data_utils.py:114

Functionpreprocess

(example)

src/data_utils.py:33

Functionprint_on_main

(*args, **kwargs)

src/dist_utils.py:63

Functionquantize_error_triton_kernel

( x_ptr, qx_ptr, error_ptr, scale_ptr, qzero_ptr, maxq_ptr, dtype_ptr, n_eleme

src/gptq_loop.py:13

Functionselect_layers

( model: nn.Module, layer_prefix: Optional[str] = "", layer_regex: str = ".*", layer_classes:

src/model_utils.py:15

Functiontokenize

(sample)

src/data_utils.py:48

Methodtokens_collected

(self)

src/gptq.py:108