Custom evaluate function to estimate the accuracy of the model. Args: model (tf.Graph or string or INC.model.TensorflowCheckpointModel):

Custom evaluate function to estimate the accuracy of the model. Args: model (tf.Graph_def): The input model graph Returns:

Evaluate function that inference the model to apply calibration or benchmarking. Args: model (tf.python.trackable.autotrackable): The mod

Custom evaluate function to inference the model for specified metric on validation dataset. Args: model (tf.keras.Model): The input m

Custom evaluate function to inference the model for specified metric on validation dataset. Args: model (tf.keras.Model): The input

Run quantization. Args: W (tensor): weight tensor. block_size (int): Execute quantization per block, block shape = [C

Get safetensors file list.

The forward wrapper.

Free memory.

Create a ConfigSet object from a single config or a list of configs. Args: fwk_configs: A single config or a list of configs.

Gather single batch from a dict. Args: data_dict (dict): data dict. idx (int): index Returns: di

Gather single batch from a list. Args: data_dict (dict): data list. idx (int): index Returns: li

Generate the valid sampling log. Args: log_path: the valid sampling log file path. Returns: the sampling min max

This is a helper method to generate an activation observer. Args: scheme (str): Quantization scheme to be used. algorithm (st

Static method that generate int32 bias for conv op. Args: bias_tensor: bias node tensor. channel_size: channel size.

Static method that generate per-channel int32 bias for matmul op. Args: bias_tensor: bias node tensor. weights_tensor

Translate the patterns defined in the yaml to internal pattern expression.

Get absorbed layers of a model. Args: model: torch model example_input: used to trace torch model. op_typ

Get all registered configurations. Returns: Dict[str, Dict[str, Dict[str, object]]]: A dictionary containing all registered confi

Return the calibrated min/max range as a tensor. Returns: jnp.ndarray: Tensor containing min and max values.

Gets desired dataloader from dlrm_main command line options. Currently, this function is able to return either a DataLoader wrapped around a

Get the default configuration of AutoRound. Args: processor_type (Optional[Union[str, torch_utils.ProcessorType]], optional): The user-sp

Generate the default awq config. Returns: the default awq config.

Generate the default HQQ config. Returns: the default HQQ config.

Get all sub-dicts that are at a specified depth in a nested dict.

Query config dict of double_quant according to double_quant_type. Args: double_quant_type (str, optional): double_quant type. Defaults to

Return the row-parallel all-reduce collective from external runtime. Returns: Callable: The tensor-model-parallel all-reduce function.

Get INC FP8 Config. Args: model: empty model. from_neuralmagic(bool, optional): whether provided from nerualmagic modelhub.

Retrieves the names of layers within each block of the model. Returns: list: A list of strings, where each string is the name of a layer

Search the matched nodes with the specified patterns. Args: patterns ([string list]): The patterns should be illustrated as below

Get the valid mixed precisions. Returns: [string list]: valid precision list.

Get concrete node names for supported operators. Args: model (keras.Model): Keras model to inspect. Returns:

Only necessary for the newly registered patched module that doesn't in _mod_default_dict. Return the module type for the module, which is used

Get all the name and children of given model.

Get new rank state_dict for world_size. Args: all_rank_state_dict (dict): {0: state_dict, 1: state_dict, ...} for all ranks. mode

Get all non-persistent buffers in the model. Args: model (torch.nn.Module): PyTorch model Returns: dict: A dictionary contai

Get op types per precision. Args: precision (string): precision name Returns: [string list]: A list composed

Get the pack function for the specified number of bits. Args: nbits (int): The number of bits. Returns: func

Get a dictionary containing the parameters and their values for the current instance. Returns: A dictionary containing the parame

Get the parent node(s) of a given node. Args: node (Node): The node whose parent(s) are to be retrieved. all_parents (bool, o

Get parent of node.

Get the post processed results.

Get quantizable ops from configs, combine fused ops as one op. Args: ops_name (list): list of op names. op_infos_from_cfgs (d

Get all quantizable ops from model. Args: model (object): input model example_inputs (dict|list|tuple|torch.Tensor): used to

Compute the quantization scale for a weight tensor. Args: orig_weight (jnp.ndarray): Weight tensor to analyze. dtype (jnp.dtype):

Get registered accelerators sorted by priority.

Get the unpack function for the specified number of bits. Args: nbits (int): The number of bits. Returns: fu

Generate the config set for WOQ tuning. Returns: the list of WOQ quant config.

Generate an input function for the Estimator.

Compute the inverse of a permutation vector. Args: perm (Tensor): A 1D tensor containing a permutation of indices. Retur

Check whether scale calculation uses cguid logic. Returns: bool: True when cguid-based scale calculation is enabled.

Check if the observer has valid calibration data. Returns: bool: True if calibrated, False if min (and internally max) are still

Check the model_path format is ckpt or not. Args: model_path (string): the model folder path Returns: string: return the ckp

Check if Habana framework is installed. Only check for the habana_frameworks package without importing it to avoid initializing lazy-mode-rel

Returns whether hpex is available.

Judge whether a module has no child-modules. Args: module: torch.nn.Module Returns: a bool: whether a module has no child-mo

Check if the Intel® Extension for TensorFlow has been installed.