Loads dataset.

Load first layer only. Args: user_model (torch.nn.Module): input model model_name (str): model name or path

Load all named parameters of module. Args: model (torch.nn.module): torch model. module_name (str): module name. path (st

Load all non-persistent buffers into the model. Args: model (torch.nn.Module): PyTorch model non_persistent_buffers (dict): A dic

Loads scale_method_config_by_mod_map from a JSON file and reconstructs the original structure. Returns: dict[str, dict[CfgStr, ScaleMethodCon

Load a tensor from bin file with given tensor name.

Load a tensor from safetensors file with given tensor name.

Build a lightweight namespace mimicking the attributes QuantLinear._setup expects.

Calculate the max metric for each ground truth. For each answer in ground_truths, evaluate the metric of prediction with this answer, and ret

Return model itself.

Perform a forward pass of the model on a single sample. Args: model (torch.nn.Module): The model to be applied. sample (Tenso

Rank the final losses of operations based on their ratio with respect to operation output norm. Returns: dict: A dictionary of

Pack tensor.

Pack tensor.

Parse configs. Args: cfgs (dict): the input configs. Returns: ops_name (list): list of op names. tune_cfg (di

Parse the quant_config to tune_cfg. Args: quant_config: a quantization configuration. model: the fp32 model to be qua

r"""Safety checker that arguments are correct.

r""" Safety checker that arguments are correct - also replaces some NoneType arguments with their default values.

Prepares a given model for quantization. Insert observers into the model so that it can monitor the input and output tensors during calibra

Prepares a given model for quantization. Will return model directly in MX quant algorithm. Args: model (torch.nn.Module)

Prepares the model for calibration. Create the `quantizer` according to the `quant_config`, and insert the observers accordingly. Ar

Prepares a given model for quantization. Args: model (torch.nn.Module): raw fp32 model or prepared model. example

Generate attention_mask from input_ids. Args: inputs (tf.Tensor): The tensor of input_ids. Returns: attention_mask (tf.Tenso

Transform a loaded quantized model. It prepares the model for inference by preparing the quantized layers. Args: model (keras.Model):

Prepare for layer-wise or blockwise quantization, including registering hooks and setting up the model path. Args: model_path (st

Preprocess the data by tokenizing.

Preprocess the quantization configuration. Args: model (torch.nn.Module): Float model to be quantized. quant_config (BaseConfig |

Perform quantization and dequantization forward pass. Args: x (Tensor): The input tensor. input_scale (Tensor): T

Deserialize a quant config from a JSON-compatible dict with class name. Args: json_obj (dict): A dict with 'quantization_type' and 'confi

Serialize a quant config to a JSON-compatible dict with class name. Args: quant_config (BaseConfig): The quantization config object to se

Quantize and dequantize the weights of a layer. Args: m (torch.nn.Module): The layer whose weights are to be quantized and dequantized

Quant and dequant tensor with group size. It's an in-place function. Args: weight: input weight scale: scale zp: zero poi

The main entry to quantize model with static mode. Args: model: a float model to be quantized. quant_config: a quantization confi

Quantizes a given float model. Args: model (torch.nn.Module): The float model to be quantized. Returns:

Execute AWQ quantization. Args: use_auto_scale (bool, optional): whether search scale. Defaults to True. use_mse_sear

Quantize graph only (without optimizing fp32 graph). Including: 1) quantize graph, 2) calibration, 3) fus

Quantize model using single config. Args: model: a model wrapped by INC TF model class. quant_config: a quantization configuratio

Quantize tensor to MX data type.

Read a SQuAD json file into a list of SquadExample.

Recursively apply a function to nested container values. Args: x (Any): Input object that can be a dict, list, tuple, or leaf value.

Removes nodes that are no longer needed for inference from the graph.

Replace the node into the internal data structure node_name_details. Args: new_node (nodedef): the nodedef object. ol

Reshape the scale for weight input channel, depthwise output channel. Args: layer (torch.nn.Module): Torch module. scale (Ten

This is neural_compressor function include tuning, export and benchmark option.

Save config mapping to json file. Args: config_mapping (dict): config mapping. qconfig_file_path (str): path to saved json file.

Save a Python dictionary to a JSON file. Args: d (dict): Data to serialize. fname (str): Destination file path.

Saves scales measured of a given model. Args: model : The measured model. d : Modules_names to configuration dictionary.

Perform a self-check to ensure that there is at least one evaluation metric registered for auto-tune. Raises: AssertionError: If

Set module with a given op name. Args: model (object): the input model. op_name (str): name of op. new_module (object): t

Set the resume_from in config.

Set the tensorboard in config.

Set the workspace in config.

Shard state_dict for current local_rank.

Strip unused nodes of the graph_def. The strip_unused_nodes pass is from tensorflow/python/tools/strip_unused_lib.py of official tensorflow r

Save this instance to a JSON file. Args: json_file_path (`str` or `os.PathLike`): Path to the JSON file in which

Track hidden states. Args: data (tensor/tuple/list): input data. Returns: tensor.

The main entry of SmoothQuant. Args: alpha (float or dict): Alpha value to balance the quantization difficulty of activation a

Update the state that need to be evaluated. Args: preds: The prediction result. labels: The reference. Defaults to No

Update the state that need to be evaluated. Args: preds: The prediction result. labels: The reference. Defaults to No

Update the state that need to be evaluated. Args: preds: The prediction result. labels: The reference. Defaults to No