Functions — github.com/Cornell-RelaxML/qtip

CodeHub MCPcopy Create free account

hub / github.com/Cornell-RelaxML/qtip / functions

Functions265 in github.com/Cornell-RelaxML/qtip

⨍ Functions 265◇ Types & classes 40 ↳ Endpoints 1

(Wr, L, cb, args, buf_cols=128, for_kernel=True)

lib/algo/ldlq.py:16

FunctionPYBIND11_MODULE

qtip-kernels/src/wrapper.cpp:489

Method__getitem__

lib/utils/data_utils.py:330

Method__getitem__

Support for backwards-compatible `past_key_value` indexing, e.g. `past_key_value[0][0].shape[2]` to get the sequence length.

model/cache_utils.py:372

Method__getitem__

Gets the cache for this layer to the device. Prefetches the next and evicts the previous layer.

model/cache_utils.py:575

Method__getitem__

Support for backwards-compatible `past_key_value` indexing, e.g. `past_key_value[0][0].shape[2]` to get the sequence length.

model/cache_utils.py:1389

(self, shards, dtype)

lib/utils/shard_model.py:46

lib/utils/data_utils.py:323

(self, input_ids)

lib/utils/gptq_data_utils.py:108

(self, *args, **kwargs)

lib/utils/graph_wrapper.py:11

( self, in_features, out_features, td_x, td_y, L, # trellis w

lib/linear/quantized_linear.py:14

(self, L=16, K=2, V=2, tlut_bits=16,

lib/codebook/bitshift.py:85

LlamaRMSNorm is equivalent to T5LayerNorm

model/llama.py:121

( self, dim=None, max_position_embeddings=2048, base=10000, device=Non

model/llama.py:146

(self, *args, **kwargs)

model/llama.py:243

(self, *args, **kwargs)

model/llama.py:255

(self, config: LlamaConfig, layer_idx: Optional[int] = None)

model/llama.py:424

(self, *args, **kwargs)

model/llama.py:665

(self, config: LlamaConfig, layer_idx: int)

model/llama.py:910

(self, config: LlamaConfig)

model/llama.py:1127

model/llama.py:1352

model/llama.py:1594

model/llama.py:1719

model/llama.py:1819

( self, backend: str = "quanto", nbits: Optional[int] = 4, axis_key: Optional[

model/cache_utils.py:227

(self, batch_size: int, max_cache_len: int, device="cpu")

model/cache_utils.py:308

(self, num_hidden_layers: Optional[int] = None)

model/cache_utils.py:366

model/cache_utils.py:550

(self, cache_config: QuantizedCacheConfig)

model/cache_utils.py:665

(self, cache_config: CacheConfig)

model/cache_utils.py:769

(self, cache_config: CacheConfig)

model/cache_utils.py:829

(self, window_length: int, num_sink_tokens: int)

model/cache_utils.py:896

( self, config: PretrainedConfig, batch_size: int = None, max_cache_len: int =

model/cache_utils.py:1089

( self, config: PretrainedConfig, batch_size: int = None, max_cache_len: int =

model/cache_utils.py:1268

(self, self_attention_cache: Cache, cross_attention_cache: Cache)

model/cache_utils.py:1380

( self, config: PretrainedConfig, batch_size: int = None, max_cache_len: int =

model/cache_utils.py:1572

( self, config: PretrainedConfig, batch_size: int = None, dtype: torch.dtype =

model/cache_utils.py:1766

( self, config: PretrainedConfig, max_batch_size: int, max_cache_len: Optional

model/cache_utils.py:1884

allows `dict(obj)` for situations where obj may be a dict or QuantizationConfigMixin

model/cache_utils.py:160

Support for backwards-compatible `past_key_value` iteration, e.g. `for x in past_key_value:` to iterate over keys and values

model/cache_utils.py:382

lib/utils/data_utils.py:327

Support for backwards-compatible `past_key_value` length, e.g. `len(past_key_value)`. This value corresponds to the number of layers

model/cache_utils.py:390

Support for backwards-compatible `past_key_value` length, e.g. `len(past_key_value)`. This value corresponds to the number of layers

model/cache_utils.py:1404

model/cache_utils.py:166

Method_dequantize

(self, qtensor)

model/cache_utils.py:797

Method_dequantize

(self, qtensor)

model/cache_utils.py:857

Method_init_weights

model/llama.py:1028

Method_quantize

(self, tensor, axis)

model/cache_utils.py:792

Method_quantize

(self, tensor, axis)

model/cache_utils.py:844

Method_sliding_update

(self, cache_position, layer_idx, key_states, value_states, k_out, v_out, max_cache_len)

model/cache_utils.py:1632

Method_static_update

(self, cache_position, layer_idx, key_states, value_states, k_out, v_out, max_cache_len)

model/cache_utils.py:1660

Functionbackward_hook

lib/utils/data_utils.py:65

Methodbatch_repeat_interleave

Repeat the cache `repeats` times in the batch dimension. Used in contrastive search.

model/cache_utils.py:1516

Methodbatch_select_indices

Only keep the `indices` in the batch dimension of the cache. Used in contrastive search.

model/cache_utils.py:1522

Methodbatch_split

Split the current instance into a list of `DynamicCache` by the batch size. This will be used by `_split_model_inputs()` in `generation.utils`

model/cache_utils.py:1488

Functionblock_LDL

(H, b, check_nan=True)

lib/utils/math_utils.py:14

Functioncalculate_ce_loss_model

(model, dataloader, start_dev, in_q, out_q)

lib/utils/finetune.py:35

Functioncalculate_logits

(model, devset, batch_size)

lib/utils/data_utils.py:350

Functioncalculate_mse_loss

(layer, dataloader, device)

lib/utils/finetune.py:16

Crop the past key values up to a new `maximum_length` in terms of tokens. `maximum_length` can also be negative to remove `maximum_length` tok

model/cache_utils.py:1482

lib/utils/data_utils.py:41

Functiondtype_from_str

lib/utils/data_utils.py:310

Methodextra_repr

model/llama.py:137

Functionfinetune_susv_e2e

(quant_model, start_dev, devset, orig_dtype, args)

lib/algo/finetune.py:376

Functionflat_to_sym

lib/utils/math_utils.py:6

(self, *args, **kwargs)

lib/utils/shard_model.py:37

(self, *args, **kwargs)

lib/utils/shard_model.py:62

(self, *args, **kwargs)

lib/utils/graph_wrapper.py:16

lib/linear/quantized_linear.py:83

(self, input, trellis, SU, SV,

lib/codebook/bitshift.py:415

(ctx, input, trellis, m, n, L, tlut_bits, K, V, lut)

lib/codebook/bitshift.py:478

(self, hidden_states)

model/llama.py:129

(self, x, position_ids)

model/llama.py:214

( self, hidden_states: torch.Tensor, attention_mask: Optional[torch.Tensor] = None,

model/llama.py:533

( self, hidden_states: torch.Tensor, attention_mask: Optional[torch.LongTensor] = None

model/llama.py:674

( self, hidden_states: torch.Tensor, attention_mask: Optional[torch.Tensor] = None,

model/llama.py:797

Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (

model/llama.py:923

( self, input_ids: torch.LongTensor = None, attention_mask: Optional[torch.Tensor] = N

model/llama.py:1152

r""" Args: labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing

model/llama.py:1384

r""" labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression

model/llama.py:1610

r""" start_positions (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for position (index) of the start of the la

model/llama.py:1734

r""" labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression

model/llama.py:1842

Functionforward_hook

(module, input, output)

lib/utils/data_utils.py:61

Methodfrom_batch_splits

This is the opposite of the above `batch_split()` method. This will be used by `stack_model_outputs` in `generation.utils`

model/cache_utils.py:511

Methodfrom_batch_splits

This is the opposite of the above `batch_split()` method. This will be used by `stack_model_outputs` in `generation.utils`

model/cache_utils.py:1501

Methodfrom_dict

Constructs a CacheConfig instance from a dictionary of parameters. Args: config_dict (Dict[str, Any]): Dictionary contain

model/cache_utils.py:113

Methodfrom_legacy_cache

Converts a cache in the legacy cache format into an equivalent `EncoderDecoderCache`.

model/cache_utils.py:1424

Methodget_decoder

model/llama.py:1378

Functionget_emb

quantize_llama/finetune_e2e_llama.py:54

Functionget_emb

eval/interactive_gen.py:91

Functionget_graph_wrapper

(cls, device=0)

lib/utils/graph_wrapper.py:7

Methodget_input_embeddings

model/llama.py:1145

Methodget_input_embeddings

model/llama.py:1363

Methodget_input_embeddings

model/llama.py:1603

Methodget_input_embeddings

model/llama.py:1727

Methodget_input_embeddings

model/llama.py:1835

Functionget_loaders

(name, nsamples=128, seed=0, seqlen=2048, model='')

lib/utils/gptq_data_utils.py:182

Methodget_max_length

Returns the maximum sequence length of the cached states. DynamicCache does not have a maximum length.

model/cache_utils.py:452

Methodget_max_length

Returns the maximum sequence length of the cached states.

model/cache_utils.py:949

Methodget_max_length

Returns the maximum sequence length of the cached states.

model/cache_utils.py:1202

← previous next →101–200 of 265, ranked by callers