Dataset used for dummy data generation. This Dataset is to construct a dataset from a specific shape. The value range is calculated from: low

Base class for TF DataLoaders. _generate_dataloader is needed to create a dataloader object from the general params like batch_size and sampl

A wrapper to construct a Neural Compressor TF Model.

Tensorflow Graph Analyzer class which implemented under singleton mode. This class provides the following API: * Analyze the graph * Anal

Config class for tf static quantization.

Config class for round-to-nearest weight-only quantization.

Config class for AUTOROUND. AUTOROUND: Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs. https://arxiv.org/a

Config class for JAX Static quantization. Static quantization applies quantization to weights offline and activations during runtime using pr

Configures a relation between a ModuleType key (from `_mod_types` dict in `common.py`) to a PatchedModule class.

Used to encapsulate measure/quant data and consumed by ModuleExtraConfig(s) 2 use cases: 1. Hold the calculated scales per op-tensor

Config for auto tuning pipeline. Examples: from neural_compressor.torch.quantization import TuningConfig tune_config = TuningConf

Config class for FP8 quantization.

Config class for GPTQ. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. https://arxiv.org/abs/2210.17323

This exception is raised when there is a problem unpickling an object, such as a security violation. Note that other exceptions may also be raise

Base model class used to obtain the model (and perform any necessary operations to make it usable)

Layer that applies static quantize-dequantize to activations.

Logger class.

This exception is raised when an unpicklable object is passed to the dump() method.

Tensor quantizer module.

Config class for fake algo.

Config class for AWQ. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration. https://arxiv.org/abs/2306.00978

Fuse Sqrt + RealDiv + Erf + AddV2 + Mul + Mul into Gelu op.

Describes the module's tensor related structure for quantization. self.num_inputs - Number inputs to be quantizeds (e.g. input of Linear).

Configuration class for quantized tensors.

Config class for smooth quantization.

Config class for keras static quantization.

Tensorflow Query Capability Class.

DataLoader dedicated for the Criteo Terabyte Click Logs dataset

Config class for JAX Dynamic quantization. Dynamic quantization applies quantization to both weights and activations during runtime. This con

OperatorConfig.

This class contains several staticmethod functions.

Config class for tf smooth quantization.

Computes and stores the average and current value

Computes and stores the average and current value

Computes and stores the average and current value

Computes and stores the average and current value

Get CPU Info.

Layer that applies dynamic quantize-dequantize to activations.

Configuration class for Half-Quadratic Quantization (HQQ). HQQ is a quantization algorithm that reduces the precision of weights and activations

Lazy import python module till use.

Config class for TEQ. TEQ: Activation-aware Weight Quantization for LLM Compression and Acceleration. https://arxiv.org/abs/2306.00978

Remove the Cast OP and set Dequantize output to B16 if the Cast OP output is BF16.

Graph Converter Class is used to generate the quantization graph.

INC Weight Only Linear.

Quantize the graph.

Convert below subgraph to Node A + LeakyRelu. Node A Node A | x | | x | |

Evaluator is a collection of evaluation functions. Note: will deprecate this class in the future. Examples: def eval_acc(model):

Freeze Value with calibration.

Config class for MX quantization.

Observer that tracks running min/max values for calibration.

The statistics printer.

Folding BatchNorm nodes into Conv.

Fuse decomposed small ops to BatchNormalization.

Config class for static quantization.

Config class for mixed-precision.

Used to encapsulate measure/quant data and consumed by PatchedModule(s) 2 use cases: 1. Holds the observers per op-tensors (MEASURE mode)

Move Squeeze op after Relu op for match fusion pattern.

Represents the meta information of a quantized tensor. Attributes: nbits (int): The number of bits used for quantization. group_s

Quantized version of nn.Linear.

Remove the unused nodes in the graph.

Define the tunable parameter for the algorithm. Example: Class FakeAlgoConfig(BaseConfig): '''Fake algo config.'''.

This class implements the bias correction graph transform. Will correct the weight and scale for *Conv2D* op weight_empirical: our task i

Not displayed in API Docs. Capture the output to file.

Get CPU Info.

Evaluation function wrapper.

Folding all the sequences only consist of const and self.supported_op_type.

Configuration class for HQQModule. Args: weight (Any): The weight quantization configuration. scale (Any): The scale quantization

Wrapper that preserves quantization config when saving Keras backbones.

Represents a quantized tensor. Example: val: torch.Tensor scale: val: torch.Tensor scale: torch.Tensor

Remove switch op if the input condition is true.

The W8A8 quantizer using PT2E.

WOQ Model Loader.

Wrap an inner module. If we do not wrap the inner module, it will not be quantized properly. Maybe we can change this behavior in the future.

BF16 node convert transformation.

Calibration class.

ConfigLoader is a generator that yields configs from a config set.

The layout conversion optimizer, convert NCHW to NHWC format. It is executed only when NCHW node exists and tensorflow version is 2.4.0 and above

Fuse Biasadd + Add into BiasAdd when the second input of Add is const node.

We introduce the CSE optimizer to optimize the nodes that contains identical op type. Case 1. Node A has three output nodes(B,C,D) and those chil

Linear wrapper to apply scale to input.