\class PluginField \brief Structure containing plugin attribute field names and associated data This information can be parsed to decode necessary pl

Temporary half-precision expression. This class represents a half-precision expression which just stores a single-precision value internally.

Temporary half-precision expression. This class represents a half-precision expression which just stores a single-precision value internally.

Internal dataclass for a single layer config. Args: name (str): Name of the layer. As seen from utilities such as `model.summary()`

Represents a graph containing nodes and tensors.

An OrderedDict[str, MetadataTuple] that maps input names to their data types and shapes. Shapes may include negative values, ``None``, or st

Functor that creates a TensorRT IBuilderConfig.

An ordered dictionary containing the result of a running a single iteration of a runner. This maps output names to NumPy arrays, and preserv

Model: the model Depends on: - RunnerSelectArgs: if guess_model_type_from_runners == True

Runs inference using TensorRT. Note that runners are not designed for production deployment and should generally be used only for protot

Helper class holding config objects for all layers to quantize.

ONNX Model Loading: loading ONNX models. Depends on: - ModelArgs - OnnxInferShapesArgs: if allow_shape_inference == True

Represents a Python script that uses the Polygraphy API.

Functor similar to EngineBytesFromNetwork, but deserializes the engine before returning.

An ordered dictionary that represents a single optimization profile that can be used to build an engine. More specifically, it is an ``O

Functor that parses an ONNX model to create a trt.INetworkDefinition.

Generates synthetic input data.

Tensor quantizer module This module uses tensor_quant or fake_tensor_quant function to quantize a tensor. And wrappers variable, moving stati

Functor that builds an ONNX-Runtime inference session.

ONNX Shape Inference: ONNX shape inference. Depends on: - OnnxLoadArgs - DataLoaderArgs: if allow_force_fallback == True

Runs inference using an ONNX-Runtime inference session.

TensorRT Builder Configuration: creating the TensorRT BuilderConfig. Depends on: - DataLoaderArgs - ModelArgs: if allow_cus

Functor that loads an ONNX model from a file.

TensorRT Network Loading: loading TensorRT networks. Depends on: - ModelArgs - TrtLoadPluginsArgs - OnnxLoadArgs: i

TensorRT Plugin Loading: loading TensorRT plugins.

ONNX-TRT Parser Flags: setting flags for TensorRT's ONNX parser Depends on: - TrtConfigArgs: If NATIVE_INSTANCENORM should be autom

Helper class for interfacing dimension constructs with Polygraphy and PyTorch.

[EXPERIMENTAL] Iteratively mark layers to run in a higher precision to find a compromise between performance and quality. `debug precision`

\class Dims3 \brief Descriptor for three-dimensional data.

An array on the GPU.

ONNX Model Saving: saving ONNX models. Depends on: - OnnxInferShapesArgs: if allow_shape_inference == True

Data Loader: loading or generating input data for inference. Depends on: - ModelArgs: if allow_custom_input_shapes == True

\class Dims2 \brief Descriptor for two-dimensional data.

A port identifies a specific layer input or output.

Create empty folders to save the original and quantized TensorFlow models and their respective ONNX models for each experiment. The following

Runs inference using custom CPU reference implementations

Maps runners to per-iteration outputs (in the form of a ``List[IterationResult]``). For example, if ``results`` is an instance of ``RunResul

Comparator Comparisons: inference output comparisons. Depends on: - CompareFuncSimpleArgs - CompareFuncIndicesArgs

Functor that creates an ONNX-GraphSurgeon graph from an ONNX ModelProto.

Tracks information about a tensor, such as format and data type.

Comparison Function: `indices`: the `CompareFunc.indices` comparison function. Depends on: - ComparatorCompareArgs

Comparison Function: `simple`: the `CompareFunc.simple` comparison function. Depends on: - ComparatorCompareArgs

Functor that deserializes an engine from a buffer.

A dropdown widget wrapper

Maps layer names to corresponding tactics. More specifically, it is an ``OrderedDict[str, Algorithm]``.

Runs inference using a TensorFlow session.

Convenience class wrapping activation regions.

\class LogStreamConsumer \brief Convenience object used to facilitate use of C++ stream syntax when logging messages. Order of base classes is LogStre

\class LogStreamConsumer \brief Convenience object used to facilitate use of C++ stream syntax when logging messages. Order of base classes is LogStre

For keeping track of workspace folder and for cleaning them up. Due to potential corruption of ONNX model conversion, the workspace is split

Functor that parses an ONNX model to create a trt.INetworkDefinition. This loader supports models with weights stored in an external location

Functor that creates a TensorFlow session that can be used for inference.

TensorRT Engine: loading or building TensorRT engines. Depends on: - ModelArgs - TrtLoadPluginsArgs - TrtLoadNetwor

A read-only view of a GPU memory region.

Creates batches of pre-processed images.

Creates batches of pre-processed images.

A dropdown widget wrapper

Global logging interface. Do **not** construct a logger manually. Instead, use ``G_LOGGER``, the global logger.

Logging: logging and debug output

TensorFlow Model Loading: loading TensorFlow models. Depends on: - ModelArgs - TfTrtArgs: if allow_tftrt == True -

Removes model bias from audio produced with waveglow

An edge in a PlanGraph graph. The src and dst are names of engine layers or regions.

Functor that loads a TensorFlow frozen model.

Creates batches of pre-processed images.

Functor that invokes a function from a Python script.

Represents a lazily loaded NumPy array. For example, large NumPy arrays may be serialized to temporary files on the disk to save memory.

To help replace torch.nn modules with quantized versions. This module is used to replace (by monkey patching) the torch.nn modules with their

TensorRT Engine: loading TensorRT engines. Depends on: - TrtLoadEngineBytesArgs - TrtLoadPluginsArgs

Represents a TensorRT algorithm variant, which can be uniquely represented by an implementation ID, tactic ID, and I/O tensor information.

Artifact Sorting: sorting artifacts into good/bad directories based on pass/fail status.

Represents a shape with min/max bounds.

Pass/Fail Reporting: reporting pass/fail status during iterative debugging.