Functions — github.com/Cornell-RelaxML/qtip

hub / github.com/Cornell-RelaxML/qtip / functions

Functions265 in github.com/Cornell-RelaxML/qtip

⨍ Functions 265◇ Types & classes 40 ↳ Endpoints 1

Methodget_max_length

(self)

model/cache_utils.py:1343

Methodget_max_length

(self)

model/cache_utils.py:1694

Methodget_max_length

Returns the maximum sequence length of the cached states.

model/cache_utils.py:2027

Methodget_output_embeddings

(self)

model/llama.py:1369

Methodget_seq_length

Returns the sequence length of the cached states. A layer index can be optionally passed.

model/cache_utils.py:723

Methodget_seq_length

Returns the sequence length of the cached states that were seen by the model.

model/cache_utils.py:1195

Methodget_seq_length

Returns the sequence length of the cached states. A layer index can be optionally passed.

model/cache_utils.py:1442

Methodget_seq_length

(self, layer_idx: Optional[int] = 0)

model/cache_utils.py:1699

Methodget_seq_length

Returns the sequence length of the cached states that were seen by the model.

model/cache_utils.py:2021

Functionget_test_tokens

(name, seed=0, seqlen=2048, model='')

lib/utils/gptq_data_utils.py:195

Methodget_usable_length

Given the sequence length of the new inputs, returns the usable length of the cache.

model/cache_utils.py:71

FunctiongpuAssert

qtip-kernels/src/inference.h:12

Functioninfer

(args, end_dev, n_layers, in_q, out_q)

lib/algo/finetune.py:351

Functionllama_arg_fn

(output, args, kwargs)

quantize_llama/finetune_e2e_llama.py:50

Functionllama_arg_fn

(output, args, kwargs)

eval/interactive_gen.py:88

Methodprepare_inputs_for_generation

( self, input_ids, past_key_values=None, attention_mask=None, inputs_e

model/llama.py:1496

Functionquantize_finetune_decoder_layer

(mixed_layer, quant_order, idx, cb, args, device, pre_orig_emb, orig_emb)

lib/algo/finetune.py:90

Functionquantize_llama_decoder

(layer, idx, cb, args, device, pre_orig_emb, orig_emb, model_config, skip_list)

quantize_llama/quantize_finetune_llama.py:67

Functionregister_detach_hook

(module)

lib/utils/data_utils.py:173

Functionregister_input_H_hook

(module, save_pfx, device)

lib/utils/data_utils.py:28

Functionregister_input_from_out_H_hook

(module, H_out)

lib/utils/data_utils.py:133

Functionregister_output_H_hook

(module, *args, **kwargs)

lib/utils/data_utils.py:53

Functionregister_output_from_in_H_hook

(module, H_in)

lib/utils/data_utils.py:92

Functionregularize_H

(H, sigma_reg)

lib/utils/math_utils.py:44

Methodreorder_cache

Saves the beam indices and reorders the cache when the tensor is back to its device.

model/cache_utils.py:597

Methodreorder_cache

Reorders the cache for beam search, given the selected beam indices.

model/cache_utils.py:1466

Methodreset

(self)

lib/utils/graph_wrapper.py:49

Methodreset

Resets the cache values while preserving the objects

model/cache_utils.py:1206

Methodreset

(self)

model/cache_utils.py:1347

Methodreset

(self)

model/cache_utils.py:1451

Methodreset

Resets the cache values while preserving the objects

model/cache_utils.py:1710

Methodreset

Resets the cache values while preserving the objects.

model/cache_utils.py:2032

Functionsample_falcon_refinedweb

(tokenizer, size=128, ctx_size=2048, nproc=1)

lib/utils/data_utils.py:255

Functionsample_rp1t

(tokenizer, size=128, ctx_size=2048, nproc=1)

lib/utils/data_utils.py:196

Functionsample_rp1t_concat

(tokenizer, size=128, ctx_size=2048, nproc=1)

lib/utils/data_utils.py:235

Functionsanity_check

(L, S, R, V)

qtip-kernels/test_decompress_matvec.py:169

Functionsave_linear

(module, path)

lib/utils/finetune.py:5

Methodseen_tokens

(self)

model/cache_utils.py:93

Methodseen_tokens

(self)

model/cache_utils.py:2046

Methodset_decoder

(self, decoder)

model/llama.py:1375

Methodset_input_embeddings

(self, value)

model/llama.py:1148

Methodset_input_embeddings

(self, value)

model/llama.py:1366

Methodset_input_embeddings

(self, value)

model/llama.py:1606

Methodset_input_embeddings

(self, value)

model/llama.py:1730

Methodset_input_embeddings

(self, value)

model/llama.py:1838

Methodset_output_embeddings

(self, new_embeddings)

model/llama.py:1372

Functionset_seed

(seed)

lib/utils/gptq_data_utils.py:9

Functionsplit_data

(X, Y, args)

lib/utils/data_utils.py:334

Functiontime_kernel

(kernel)

qtip-kernels/test_decompress_matvec.py:42

Methodto_json_file

Save this instance to a JSON file. Args: json_file_path (`str` or `os.PathLike`): Path to the JSON file

model/cache_utils.py:134

Methodto_legacy_cache

Converts the `EncoderDecoderCache` instance into its equivalent in the legacy cache format.

model/cache_utils.py:1411

Functionunpack_quip

(module, saved_layer)

lib/utils/data_utils.py:281

Methodupdate

Updates attributes of this class instance with attributes from `kwargs` if they match existing attributes, returning all the unused k

model/cache_utils.py:178

Methodupdate

Updates the cache with the new `key_states` and `value_states` for the layer `layer_idx`. Parameters: key_states (`torch.

model/cache_utils.py:604

Methodupdate

( self, key_states: torch.Tensor, value_states: torch.Tensor, layer_idx: int,

model/cache_utils.py:680

Methodupdate

Updates the cache with the new `key_states` and `value_states` for the layer `layer_idx`. Parameters: key_states (`torch

model/cache_utils.py:953

Methodupdate

Updates the cache with the new `key_states` and `value_states` for the layer `layer_idx`. It is VERY important to index using a tenso

model/cache_utils.py:1147

Methodupdate

( self, key_states: torch.Tensor, value_states: torch.Tensor, layer_idx: int,

model/cache_utils.py:1296

Methodupdate

( self, key_states: torch.Tensor, value_states: torch.Tensor, layer_idx: int,

model/cache_utils.py:1668

Methodupdate

Updates the cache with the new `key_states` and `value_states` for the layer `layer_idx`. It is VERY important to index using a tenso

model/cache_utils.py:1938

Methodupdate_conv_state

( self, layer_idx: int, new_conv_state: torch.Tensor, cache_position: torch.LongTensor )

model/cache_utils.py:1805

Methodupdate_ssm_state

(self, layer_idx: int, new_ssm_state: torch.Tensor)

model/cache_utils.py:1817

Methodvalidate

Validates if the arguments passed are correct

model/cache_utils.py:247

Methodvalidate

Validates if the arguments passed are correct

model/cache_utils.py:313

Functionwrap_tokenizer

(tokenizer, x, ctx_size, truncate=True)

lib/utils/data_utils.py:188

← previous201–265 of 265, ranked by callers