Generates a string that would invoke the type specified in type_str with the specified arguments. Skips keyword arguments that are set to

Align two plans by their layers. When comparing the layers of two engine plans, we want to find pairs of layers, one from each plan, that cor

Merge reference and target profile results into a single row

Extract integer and fractional parts. \param arg number to decompress \param iptr address to store integer part at \return fractional part template<ty

Extract integer and fractional parts. \param arg number to decompress \param iptr address to store integer part at \return fractional part template<ty

Mimic np.round as in voxel generator in spconv implementation

* Exchange the block from nonopt_start to nonopt_end with the block * from nonopt_end to opt_end (keeping the same order of arguments * in each bloc

Power function. \param base first argument \param exp second argument \return \a base raised to \a exp template<typename T,typename U> typename enable

Creates an untyped device array of the specified shape. Args: shape (Tuple[int]): The initial shape of t

Render dot graph to an external file using the specified format.

\brief Calculate layer timing info for the current optimization profile in IExecutionContext and update the profiler after one iteration of inference

Saves the Excel file. Once the file is saved, it cannot be updated using the same instance of the class. A new instance may

Extracts a subgraph from the main graph and saves it to disk. Args: graph (onnx_graphsurgeon.Graph): The parent/main graph.

\brief Set the axis along which softmax is computed. Currently, only one axis can be set. The axis is specified by setting the bit corresponding to t

\brief Set the LRN beta value. The valid range is [0.01, 1e5f]. \see getBeta()

\brief Set the bias weights for the convolution. Bias is optional. To omit bias, set the count value of the weights structure to zero. The bias is a

\brief Add a calibration profile. Calibration optimization profile must be set if int8 calibration is used to set scales for a network with runtime d

\brief Sets the DLA core used by the network. Defaults to -1. \param dlaCore The DLA core to execute the engine on, in the range [0,getNbDlaCores()).

\brief Replace an input of this layer with a specific tensor. \param index the index of the input to modify. \param tensor the new input tensor Exce

\brief Set the scale mode. \see getMode()

\brief Set the output type of this layer Setting the output type constrains TensorRT to choose implementations which generate output data with the gi

Set plugin namespace

Set plugin namespace

Set plugin namespace

Set plugin namespace

Set plugin namespace

Set plugin namespace

Set plugin namespace

Set plugin namespace

\brief Set the multi-dimension post-padding for pooling. The end of the input will be padded by this number of elements in each dimension. Padding va

\brief Set the multi-dimension pre-padding for pooling. The start of the input will be padded by this number of elements in each dimension. Padding v

\brief Set the computational precision of this layer Setting the precision allows TensorRT to choose an implementation which run at this computationa

\brief Set the reshaped dimensions. \param dimensions The reshaped dimensions. Two special values can be used as dimensions. Value 0 copies the cor

\brief Set memory address for given input or output tensor. \param tensorName The name of an input or output tensor. \param data The pointer (void*)

Sets the input metadata for the calibrator. This is passed along to the data loader and is also used for input d

Create subdirectory for models with different config(e.g. kv cache)

Sets the active optimization profile for this runner. The runner must already be active (see ``__enter__()`` or ``activate()``).

Sets the return device of the return via to(). Device name should be the same as torch devices: cuda, cpu, etc. This is used in our m

Set up a command-line argument parser. Args: subparsers (argparse.SubParsers): A subparser group fro

Sine function. \param arg function argument \return sine value of \a arg template<typename T> typename enable<expr,T>::type sin(T arg) { return functi

Sine function. \param arg function argument \return sine value of \a arg template<typename T> typename enable<expr,T>::type sin(T arg) { return functi

Returns the size in bytes of the specified data type.

Create a dictionary of important attributes of the engine plan.

Invokes the ONNX parser's `supports_model` on the specified model. Args: path (str): The path to the ONNX model. Returns:

Convert plan-graph to dot format

Creates a TensorRT IOptimizationProfile based on the values set in this Profile. Args: builder (trt.Builder):

Create yaml serialization Some attributes need special treatment to have human readable form, including amax, axis.

Runs basic whitespace cleaning and splitting on a piece of text.

Adds a fake quantization operation. Depending on value of per_channel, this operation may do global quantization or per channel quantization.

Args: allow_custom_outputs (bool): Whether to allow marking custom output tensors. Defaul

Args: allow_shape_inference (bool): Whether to allow shape inference when saving models.