Pack the tensor with numpy.

Pack the tensor with torch. Args: raw_tensor (tensor): raw tensor. Returns: tensor: packed tensor.

Parse the input pbdef to get the node name and node mapping. Returns: the dict that key is node name while the value is nodeDef.

Replaces the module with patched module according to mod_dict. Args: mod (nn.module): The module that will be replaced with a patched mod

Initialization for W4A8 usage.

Convert a dense-like layer instance for static quantization. Args: orig (keras.layers.Layer): Original layer instance.

Quant and dequant tensor per channel. It is an in-place op. Args: weight : input weight bits (int, optional): bits. Defaults to 4

Quantizes a given float tensor using the specified tensor quantization configuration. Args: float_tensor (torch.Tensor): The floa

Reconstruct a saved_model. Args: graph_def: The input graph_def. func: The concrete function get from the original saved_model.

Recover ipex model from JSON file. Args: model (object): fp32 model need to do quantization. json_file_path (json): configura

Register a model to TFSlimNetsFactory. Args: name (str): name of a model. model_func (_type_): model that built from

Move the input channel to the last dimension. Args: layer_name (str): Layer name. Returns: Tensor: The reshaped weight

this code is copied from vllm RowParallelLinear forward method

Set the node's attr which data type is bool.

Set the node's attr which data type is int.

Set the node's attr which data type is string.

Set the baseline value for auto-tune. Args: baseline (float): The baseline value to be set.

Args: weight (torch.Tensor): weight tensor. tp_size (int): tensor parallel size. tp_rank (int): tensor parallel rank.

Serializes this instance to a JSON string. Args: use_diff (bool, optional): If True, only the difference between the config insta

Train function.

Log the end of a trial.

Log the start of a trial.

Log the end of the tuning process.

Log the start of the tuning process.

Unpack tensor.

Add the predictions and labels. Args: predicts: The predictions. labels: The labels corresponding to the predictions.

Add the predictions and labels. Args: predicts: The predictions. labels: The labels corresponding to the predictions.

Add the predictions and labels. Args: predicts: The predictions. labels: The labels corresponding to the predictions.

Pack weight and scale when saving.

Runs basic whitespace cleaning and splitting on a piece of text.

Convert a box in [ymin, xmin, ymax, xmax] format to COCO format. This is a utility function for converting from our internal [ymin, xmin, yma

Convert a box in [ymin, xmin, ymax, xmax] format to COCO format. This is a utility function for converting from our internal [ymin, xmin, yma

Convert a box in [ymin, xmin, ymax, xmax] format to COCO format. This is a utility function for converting from our internal [ymin, xmin, yma

Compresses mask using Run-length encoding provided by pycocotools. Args: masks: uint8 numpy array of shape [mask_height, mask_width] with v

Compresses mask using Run-length encoding provided by pycocotools. Args: masks: uint8 numpy array of shape [mask_height, mask_width] with v

Compresses mask using Run-length encoding provided by pycocotools. Args: masks: uint8 numpy array of shape [mask_height, mask_width] with v

Init GPTQ.

Initialize a BaseModel. Args: model (object): raw model format. For Tensorflow model, could be path to frozen pb file,

Return the string representation of the QTensor object.

Insert the Mul node after the activation before the weight node. Args: scale: smooth scale with the shape (ic,) input

Retrieves sq info to absorb the scale to the layer at output channel. Args: model (QuantizationInterceptionModule): a prepared model.

Build weight-only quantization model.

Compute batch min/max from inputs, respecting mask. Args: inputs (jnp.ndarray): Input tensor. mask (Optional[jnp.ndar

Create new Const node.

Return (min, max) representable values for dtype, cast to out_dtype.

Dump the whole model's OPs statistics information for analysis.

r"""Replace characters that aren't in the alphabet and append "_" to token. Apply three transformations to the token: 1. Replace underline char

Find Relu node algorithm to identify the positive input.

Freeze requantization ranges after doing quantization.

Expand a single config into a list of configs. Args: config (BaseConfig): Configuration to expand. Returns:

Create set of characters that appear in any element in the iterable.

Generate the calibration data.

Create a sampler based on dataset capabilities. Args: dataset: Dataset object to inspect. distributed (bool): Whether

Create a list of subtokens in decreasing order of frequency. Args: token_counts: dict mapping str tokens -> int count alphabet: set of char

Obtain all the modules that could be hooked based on given op_types.

Identify the layers which can be smooth quantized. Args: op_types (list): The operation types to be smooth quantized.

Get the n-best logits from a list.

Retrieve an example input from the dataloader or return the pre-stored example inputs. Returns: Union[torch.Tensor, None]: The

Get parameters of the format. Allowed formats: - intX: 2 <= X <= 32, assume sign-magnitude, 1.xxx representation - floatX/fpX:

Get the print node name of the fp32 graph.

The version 2 function that get graph from the original keras model. Args: saved_model_dir (string): model path of a temporary saved_mode

The version 3 function that get graph from saved_model. Args: model (string or tf.keras.Model): model path or tf.keras.Model object.

Map a Mode enum value to a human-readable log message. Args: mode (Mode): Execution mode enum. Returns: str | None: Log mess

Valid only for floats that define NaN.

Get the max values per input channel. Args: tensor_data: The input tensors percentile: The percentile of calibration

Valid for all float formats.

Calculates n-grams. Args: n: which n-grams to calculate text: An array of tokens Returns: A set of n-grams

Extracts all n-grams up to a given maximum order from an input segment. Args: segment: text segment from which n-grams will be extracted. m

Split local configs into op-type and op-name mappings. Returns: tuple[dict, dict]: Mapping of op types to configs and op names to