Move the QTensor object to a new device or new dtype.

Output log with the info level.

The main entry of auto-tune.

Set the attribute data type.

The main entry to quantize model. Args: model: a fp32 model to be quantized. quant_config: single or lists of quantization config

Output log with the debug level.

Output log with the info level.

Get the original node name from input string. Args: node_name: input node's name in string Returns: node's n

Convert the prepared model to a quantized model. Args: model (torch.nn.Module): torch model quant_config (BaseConfig, optional):

Prepare the model for calibration. Insert observers into the model so that it can monitor the input and output tensors during calibration. A

Create a nodedef object. Args: op (string): op type name (string): op name inputs (string list): op's inp

Copy the specified attr value to node. Args: node (nodedef): a nodedef object key (string): string name a

Check whether version1 is greater than version2 or is equal to it.

Inserts one node into the new graph.

Read a pickled object representation from the open file. Return the reconstituted object hierarchy specified in the file.

Return control_input name, non-control_input node name.

Output log with the warning level (Alias of the method warn).

Remove the user specified node by its name. Args: node_name (string): node name string. Returns: [bool]: Tru

Abstract method of model saving.

Base Interface that need to be implemented by each sub class.

Create constant node. Args: name (string): op name value (np.array): input data dtype (datatype): data ty

Add the node into the internal data structure node_name_details. Args: new_node (nodedef): the nodedef object. start_

Set the node's attr which data type is int list.

Updates attributes of this class instance with attributes from `kwargs` if they match existing atributtes, returning all the unused kwargs.

Analyze the input graphdef and return the list contains each node's input/outputnode names. Args: input_graph_def ([graphdef]): g

Dump the current model's graphdef. Returns: [graphdef]: A graphdef object

Output log with the warning level (Alias of the method warn).

Set the node's attr which data type is bool.

Set the node's attr which data type is int list.

Set the node's attr which data type is string.

This is neural_compressor function include tuning, export and benchmark option.

Change device for all values.

Do nothing.

Load quantized model. 1. Load INC quantized model in local. case 1: WOQ from neural_compressor.torch.quantization import load

Public interface for query the nodes aggregation status. Args: patterns (string list): Please check the _search_patterns definiti

Set custom configuration based on the global configuration object. Args: operator_name_or_list (Union[List, str, Callable]): Spec

Merge data with outer dimension batch size.

Automatically detects and selects the appropriate accelerator. Force use the cpu on node has both cpu and gpu: `INC_TARGET_DEVICE=cpu` python mai

Add Dequantize node after the quantized node.

Converts list of int subtokens ids into a string.

Find relu node algorithm to identify the positive input.

Get the current device name.

Do nothing.

Output log with the trace level.

Execute the fusion from small ops to Gelu.

Save a configuration object to the directory `save_directory`, so that it can be re-loaded using the [`~PretrainedConfig.from_pretrained`] cla

Add quantize down nodes.

Encodes a string into a list of int subtoken ids.

Always return True.

Returns a dictionary containing the state of the module. The state dictionary contains the weights of the `q_weight` attribute. If th

Remove the redundant Quantize/QuantizeV2 nodes.

Quantize weight node.

Write a pickled representation of obj to the open file.

Save FP8 model. Args: model (torch.nn.Module): fp8 model object. checkpoint_dir (str, optional): path to checkpoint. Defaults to

The main entry of auto-tune. Args: model (torch.nn.Module): The model to be quantized. tune_config (TuningConfig): The configurat

Set the node's attr which data type is int.

Get the accelerator name.

Dequantizes the QTensor into a float tensor. Args: q_weight (QTensor): The quantized weight tensor. Returns:

Static method that get the valid node name from input name. Args: node_name (string): node name defined in the input field.

Remove observers and finalize quantized call path. Returns: None: Cleans up observers and sets quantized call.

Return a quantized Keras model according to the given configuration. Args: model (keras.Model): FP32 Keras model to be quantized.

Register transformers API.

Create quantization variables for activations. Returns: None: Initializes quantization variables.

Converts the QTensorMetaInfo object to a dictionary. Returns: dict: A dictionary representation of the QTensorMetaInfo object.

Trace and freeze a model using TorchScript, handling various input formats and devices. Args: model (torch.nn.Module): The mod

Compute activation scale and finalize static quantization. Uses fixed_range if set, otherwise reads from the calibration observer. R

Get the current device.

Get HPU used memory: MiB.

Set the random seed in config.

Get the op list.

Attach observer layers for calibration. Skipped when fixed_range is set, as no calibration data is needed. Returns: None

Get module from model by key name. Args: model (torch.nn.Module): original model key (str): module name to be replaced

Return the recommended accelerator based on device priority.

Replace the specified const node with another one. Args: new_const_node (NodeDef): node name string. target_node (lis

Check whether transformers is imported.

Do nothing.