MCPcopy Create free account

hub / github.com/OpenGVLab/EfficientQAT / functions

Functions110 in github.com/OpenGVLab/EfficientQAT

⨍ Functions 110◇ Types & classes 25
↓ 37 callersFunctionregister_conv_template
Register a new conversation template.
deita_dataset/conversation.py:259
↓ 10 callersFunctionrank0_print
(*args)
deita_dataset/train.py:73
↓ 5 callersMethodappend_message
Append a new message.
deita_dataset/conversation.py:197
↓ 5 callersFunctionload_data
(dataset_name)
datautils_e2e.py:99
↓ 5 callersFunctiontest_ppl
(model, tokenizer, datasets=['wikitext2'],ppl_seqlen=2048)
datautils_block.py:173
↓ 4 callersFunctionload_quantized_model
(model_path, wbits, group_size)
quantize/int_linear_real.py:174
↓ 4 callersMethodupdate_data
(self, idx, new_data)
datautils_block.py:271
↓ 4 callersFunctionupdate_dataset
(layer, dataset, dev, attention_mask, position_ids)
quantize/block_ap.py:22
↓ 3 callersMethod_get_file_path
(self, idx)
datautils_block.py:255
↓ 3 callersFunctionformat_dataset
(dataset, dataset_format)
datautils_e2e.py:150
↓ 3 callersFunctionset_op_by_name
(layer, name, new_module)
quantize/utils.py:104
↓ 3 callersFunctionset_quant_state
(model, weight_quant: bool = False)
quantize/utils.py:69
↓ 2 callersFunctionclamp_ste
(x: torch.Tensor, min, max)
quantize/quantizer.py:16
↓ 2 callersMethodcopy
(self)
deita_dataset/conversation.py:231
↓ 2 callersFunctiondequant_dim0
(qweight, bits, maxq, infeatures, outfeatures)
quantize/triton_utils/kernels.py:219
↓ 2 callersFunctiondequant_dim1
(qweight, bits, maxq, infeatures, outfeatures)
quantize/triton_utils/kernels.py:234
↓ 2 callersFunctionget_conv_template
Get a conversation template.
deita_dataset/conversation.py:269
↓ 2 callersFunctionget_loaders
( name, tokenizer, train_size=128, val_size=64,seed=0, seqlen=2048, test_only=False )
datautils_block.py:158
↓ 2 callersFunctionget_major_and_minor_from_version
(full_version)
main_e2e_qp.py:33
↓ 2 callersFunctionget_named_linears
(module, type)
quantize/utils.py:100
↓ 2 callersMethodget_prompt
Get the prompt for generation.
deita_dataset/conversation.py:52
↓ 2 callersFunctionis_ipex_available
()
main_e2e_qp.py:32
↓ 2 callersFunctionpreprocess
( sources, tokenizer: transformers.PreTrainedTokenizer, conv_template = "vicuna-1.1", mask_use
deita_dataset/train.py:79
↓ 2 callersMethodprune_configs
(self, kwargs)
quantize/triton_utils/custom_autotune.py:107
↓ 2 callersFunctionround_ste
Implement Straight-Through Estimator for rounding operation.
quantize/quantizer.py:10
↓ 2 callersFunctionset_quant_parameters
(model, requires_grad)
quantize/utils.py:39
↓ 2 callersFunctionset_weight_parameters
(model, requires_grad)
quantize/utils.py:25
↓ 1 callersMethod__init__
(self, raw_data, tokenizer: transformers.PreTrainedTokenizer, conv_template = "vicuna-1.1", mask_user = True)
deita_dataset/train.py:263
↓ 1 callersMethod__len__
(self)
datautils_block.py:258
↓ 1 callersMethod_bench
(self, *args, config, **meta)
quantize/triton_utils/custom_autotune.py:55
↓ 1 callersMethod_initialize_data_on_disk
(self)
datautils_block.py:249
↓ 1 callersFunctionampscaler_get_grad_norm
(parameters, norm_type: float = 2.0)
utils.py:10
↓ 1 callersMethodbackward
(ctx, grad_output)
quantize/utils.py:90
↓ 1 callersFunctionblock_ap
( model, args, trainloader, valloader, logger=None, )
quantize/block_ap.py:33
↓ 1 callersFunctionevaluate
Note: evaluation simply move model to single GPU. Therefor, to evaluate large model such as Llama-2-70B on single A100-80GB, please acti
main_block_ap.py:23
↓ 1 callersMethodfake_quant
(self, x)
quantize/quantizer.py:58
↓ 1 callersFunctionget_accelerate_model
(args, checkpoint_dir)
main_e2e_qp.py:230
↓ 1 callersFunctionget_c4
(tokenizer, train_size, val_size, seed, seqlen, test_only)
datautils_block.py:45
↓ 1 callersFunctionget_last_checkpoint
(checkpoint_dir)
main_e2e_qp.py:371
↓ 1 callersFunctionget_redpajama
(tokenizer, train_size, val_size, seed, seqlen)
datautils_block.py:117
↓ 1 callersFunctionget_wikitext2
(tokenizer, train_size, val_size, seed, seqlen, test_only)
datautils_block.py:13
↓ 1 callersFunctionmain
()
main_block_ap.py:62
↓ 1 callersFunctionmain
()
model_transfer/real_to_fake.py:6
↓ 1 callersFunctionmain
()
model_transfer/efficientqat_to_others.py:8
↓ 1 callersFunctionmain
()
model_transfer/fp32_to_16.py:7
↓ 1 callersFunctionmake_data_module
Make dataset and collator for supervised fine-tuning or continue pre-train.
datautils_e2e.py:95
↓ 1 callersFunctionmake_supervised_data_module
Make dataset and collator for supervised fine-tuning.
deita_dataset/train.py:335
↓ 1 callersMethodpack
(self, linear, scales, zeros, g_idx=None)
quantize/int_linear_real.py:99
↓ 1 callersFunctionprint_trainable_parameters
Prints the number of trainable parameters in the model.
main_e2e_qp.py:324
↓ 1 callersFunctionquant_inplace
(model)
quantize/utils.py:75
↓ 1 callersFunctionquant_parameters
(model)
quantize/utils.py:46
↓ 1 callersMethodrun
(self, *args, **kwargs)
quantize/triton_utils/custom_autotune.py:79
↓ 1 callersMethodset_quant_state
(self, weight_quant: bool = False)
quantize/int_linear_fake.py:53
↓ 1 callersFunctionsmart_tokenizer_and_embedding_resize
Resize tokenizer and embedding. Note: This is the unoptimized version that may make your embedding size not be divisible by 64.
main_e2e_qp.py:344
↓ 1 callersFunctiontrain
()
main_e2e_qp.py:385
↓ 1 callersFunctiontrain
()
deita_dataset/train.py:365
↓ 1 callersFunctiontrainable_parameters
(model)
quantize/utils.py:54
↓ 1 callersFunctiontrainable_parameters_num
(model)
quantize/utils.py:61
↓ 1 callersMethoduse_fake_quantization
(self, del_quant=False,transpose=False)
quantize/int_linear_real.py:79
↓ 1 callersFunctionweight_parameters
(model)
quantize/utils.py:32
Method__call__
(self, loss, optimizer, clip_grad=None, parameters=None, create_graph=False, update_grad=True,retain_graph=Fal
utils.py:31
Method__call__
(self, instances: Sequence[Dict])
datautils_e2e.py:28
Method__getitem__
(self, idx)
datautils_block.py:261
Method__getitem__
(self, i)
deita_dataset/train.py:287
Method__getitem__
(self, i)
deita_dataset/train.py:321
Method__init__
(self)
utils.py:28
Method__init__
(self, size, seqlen, hidden_size, batch_size, dtype, cache_path='./cache/block_training_data', off_load_to_dis
datautils_block.py:232
Method__init__
(self, *args, **kwargs)
quantize/utils.py:9
Method__init__
( self, n_bits: int = 8, group_size=None, weight=None, )
quantize/quantizer.py:24
Method__init__
(self, module, dataset)
quantize/block_ap.py:80
Method__init__
( self, org_module: nn.Linear, wbits=4, group_size=64 )
quantize/int_linear_fake.py:15
Method__init__
( self, bits, group_size, infeatures, outfeatures, bias,
quantize/int_linear_real.py:29
Method__init__
( self, fn, arg_names, configs, key, reset_to_zero, pr
quantize/triton_utils/custom_autotune.py:16
Method__init__
(self, raw_data, tokenizer: transformers.PreTrainedTokenizer, conv_template = "vicuna-1.1", mask_user = True)
deita_dataset/train.py:298
Method__len__
(self)
deita_dataset/train.py:284
Method__len__
(self)
deita_dataset/train.py:318
Method_hook
(args)
quantize/triton_utils/custom_autotune.py:38
Methodadd_block
(self, block)
quantize/utils.py:13
Functionautotune
(configs, key, prune_configs_by=None, reset_to_zero=None, nearest_power_of_two=False)
quantize/triton_utils/custom_autotune.py:135
Methodchange_n_bits
(self, n_bits)
quantize/quantizer.py:53
Functioncreate_logger
(output_dir, dist_rank=0, name='')
utils.py:54
Functiondecorator
(fn)
quantize/triton_utils/custom_autotune.py:136
Functiondequant_kernel_dim0
dequant the quantized tensor to fp tensor B is of shape (M/(32//bits), N) int32 C is of shape (M, N) float16
quantize/triton_utils/kernels.py:83
Functiondequant_kernel_dim1
dequant the quantized tensor to fp tensor B is of shape (M, N/(32//bits)) int32 C is of shape (M, N) float16
quantize/triton_utils/kernels.py:169
Methoddict
(self)
deita_dataset/conversation.py:245
Functionextract_alpaca_dataset
(example)
datautils_e2e.py:87
Methodforward
(self, hidden_states: torch.Tensor, attention_mask: Optional[torch.Tensor] = None, pos
quantize/utils.py:16
Methodforward
(ctx, input, threshold)
quantize/utils.py:83
Methodforward
(self, x: torch.Tensor)
quantize/quantizer.py:77
Methodforward
(self, inp, **kwargs)
quantize/block_ap.py:88
Methodforward
(self, input: torch.Tensor)
quantize/int_linear_fake.py:39
Methodforward
(self, x)
quantize/int_linear_real.py:157
Functiongroup_texts
(examples)
datautils_e2e.py:177
Functionhadamard248_kernel_config_pruner
The main purpose of this function is to shrink BLOCK_SIZE_* when the corresponding dimension is smaller.
quantize/triton_utils/custom_autotune.py:174
Methodkernel_call
()
quantize/triton_utils/custom_autotune.py:65
Methodload_state_dict
(self, state_dict)
utils.py:50
Functionmake_inputs_require_grad
(module, input, output)
main_e2e_qp.py:303
Functionmatmul248_kernel_config_pruner
The main purpose of this function is to shrink BLOCK_SIZE_* when the corresponding dimension is smaller.
quantize/triton_utils/custom_autotune.py:144
Methodon_evaluate
(self, args=None, state=None, control=None, model=None, **kwargs)
main_e2e_qp.py:433
Methodpost_init
(self)
quantize/int_linear_real.py:75
next →1–100 of 110, ranked by callers