Check whether SavedModel functions declare implementation attributes. Args: saved_model_proto: Loaded SavedModel protocol buffer. Re

Convert Dequantize + Op + QuantizeV2 into QuantizedOps.

Build an Example proto for an example. Args: filename: string, path to an image file, e.g., '/path/to/example.JPG' image_buffer: string, JP

Count number of times subtokens appear, and generate new subtokens. Args: token_counts: dict mapping tokens to the number of times they appear

Return token counts of words in the files. Samples file_byte_limit bytes from each file, and counts the words that appear in the samples. The sam

Initializes an HF or HF-compatible PreTrainedModel from scratch inside HFLM, using the kwargs passed into self.__init__(). Also handl

Helper method during initialization. Create a tokenizer object corresponding to the correct tokenizer for value of `pretrained`, or u

Detect physical CPU core count using psutil. Returns: int: Number of physical CPU cores.

Get the dtype value of the node.

Dump min-max per channel information; the min-max values will be saved in the input_maxes attribute. Args: calibration_method

Evaluates model. Computes and prints AUROC Args: model (torch.nn.Module): model for evaluation. eval_dataloader (DataLoader)

Exports the torch model to the packed checkpoint with original HF naming. The packed checkpoint will be consumed by the TensorRT-LLM unified conv

For the given weight attr of the sub_module, export the quantization info of it. The export includes converting weight tensor to correct quantize

Export information required for SmoothQuant including scales and min/max values. Args: absorb_to_layer (dict): A dictionary ma

Computes the LCS-based F-measure score. Source: http://research.microsoft.com/en-us/um/people/cyl/download/papers/ rouge-working-note-v1.3.1.pdf

Return op_name and op_type from key, such as "('transformer.h.0.attn.k_proj', 'Linear')".

Return a bucketed list of subtokens that are filtered by count. Args: subtoken_counts: defaultdict mapping subtokens to their counts min_co

Filter out unquantizable ConcatV2 Ops based on the positive input rule.

OOB filter out unquantizable ConcatV2 OPs by checking the control flow rule.

Build a list of human-readable labels. Args: synsets: list of strings; each string is a unique WordNet ID. synset_to_human: dict of synset

Build a list of all images files and labels in the data set. Args: data_dir: string, path to the root directory of images. Assumes that

Calculate values of end node of constant node sequence. there may be layers whose inputs are all constant in the graph, like: const

Expand a list of configs into a single list. Args: fwk_configs (List[BaseConfig]): Configurations to expand. Returns:

Fusing Batch Normalization.

Generate candidate subtokens ordered by count, and new max subtoken length. Add subtokens to the candidate list in order of length (longest subtoke

Generate nodes hash md5 data.

Generate the temporary file names.

Generate the calibration data.

Generate the calibration data.

Yield batches from the dataset using the configured sampler. Args: dataset: Dataset to iterate. batch_size (int): Bat

Yield data.

Yield data.

Yield data.

Yield data.

Yield data.

Yield data.

Yield data.

Yield data.

Yield data.

Yield data.

Yield data.

Generate subtoken vocabulary close to the target size.

Generate a unique identifier for a trial record. Returns: str: Unique identifier string.

Get absorbed layer per block from absorbed layer dict. Args: model (torch.nn.Module): input model absorb_layer_dict (dict): The l

Get absorbed layer per block. Args: model (torch.nn.Module): input model. example_inputs (tensor/tuple/dict, optional): used to t

Returns the kwargs needed to apply `accelerate` in `AutoModel.from_pretrained`.

Helper method during initialization. Determines the backend ("causal" (decoder-only) or "seq2seq" (encoder-decoder)) model type to be

Get the model device. Returns: torch.device: The device on which the model is located.

Get the model device. Returns: str: Model device.

The version 2 function that get graph from the original keras model. Args: model (string or tf.keras.Model): model path or tf.keras.Model

Gt quantization config of current module. 1. INC weight-only quantization model, quantization_config will be structured in module level like:

Detect the number of CPU sockets available. Returns: int: Number of CPU sockets detected.

Calculate the losses for all alpha values given an input. Returns: dict: A dictionary of operation-wise loss values with respe

Calculate the losses for all alpha values given an input in blockwise tuning mode. Returns: dict: A dictionary of blockwise lo

Get op list by recursive sorting the graph.

Get the output_node_name and index mapping.

Get weight archive file of model.

Get the configuration for the current runtime. If there's no matched configuration in the input yaml, we'll use the configuration of

Get the configuration for the current runtime. If there's no matched configuration in the input yaml, we'll use the `default` field o

Split the node name into two parts. Returns: Pure string name without suffix Index of the node

Get scale for weight. Args: weight (tensor): input weight q_group_size (int, optional): how many elements share one scale/zp. Def

Get the weight scale and zero-point. Returns: Tensor: The scale of the weight.

Load and cache weight tensors needed for smooth quantization.

Collect attention and ffn blocks from detect result.

Check the start_node if has positive input.

Returns tokenized answer spans that better match the annotated answer.

Inference the model to apply calibration. Args: sampling_graph_def: The temporary graph_def for inference.

Run the calibration on the input graph. Args: model(TensorflowBaseModel): input TensorflowBaseModel

Run the calibration on the input graph.