Format parameter count in terms of K (1000)

Exact match of math if and only if: 1. numerical equal: both can convert to float and are equal 2. symbolic equal: both can convert to sy

Count total, classifier, and non-classifier trainable parameters in a PyTorch model. Returns values in terms of K (1000) parameters.

Estimates and processes gradients using batch-wise computation. Returns a dictionary of processed gradients. Args: model: Th

:param adapter_name: options: 'default' :param reconstr_type: options: 'svd'

Generate prompt from data point.

device is executing device origin_device is the device where tensor is saved after forward

Use PyTorch's SVD which can utilize GPU acceleration

Check if the layer is part of classifier based on common naming patterns

Tokenize the prompt.

Clean Numbers in the given string >>> _clean_numbers(None, "Hello 123") 'Hello 123' >>> _clean_numbers(None, "Hello 1234") 'Hell

Tokenize a list of strings.

Split data into batches.

Main evaluation function for commonsense tasks.

Create a directory structure for the current training run.

Create a directory structure for the current training run.

Extract the answer from model output based on dataset type.

:param adapter_name: options: 'default' :param reconstr_type: options: 'svd'

Generate and tokenize prompt with proper labels.

Generate prompt in the standard format.

ensure same device for input and module

Get the module name list of the leaf nodes of the module tree, and stop recursing when the specified node(no_split_module_class) is

Creates a hook to record the gradients of a model's parameters into a dictionary. Args: model (torch.nn.Module): The model whose gra

Calculates the ratio of original type size to quantized type size. Args: origin_type (str, optional): The data type of the original

Load and preprocess the dataset.

The offload conditions are: the tensor is on cuda, and the number of bytes occupied by the tensor is equal to the theoretical number of b

Preprocess the data by tokenizing.

Register the hook in the appropriate module

remove hook registered by __enter__

Offload part of model to cpu. Args: model (torch.nn.Module): Model to be offloaded named_grads (dict): Contains the n

Offload model during forward and backward. Args: model (torch.nn.Module): The model to which the hook will be applied.

Initializes the ModelOffloadHookContext to manage offloading of model computations and saved tensors. Args: model (torch

Offload gradient to cpu Args: model (torch.nn.Module): The model whose gradients will be offloaded. record_dict (dict

get full name list of all submodule. result is self.