Functions — github.com/Mega4alik/ollm

CodeHub MCPcopy Create free account

hub / github.com/Mega4alik/ollm / functions

Functions149 in github.com/Mega4alik/ollm

⨍ Functions 149◇ Types & classes 47

(self, model_dir, adapter_dir=None, device="cuda:0", logging=True, multimodality=False)

src/ollm/inference.py:140

Function_attn_fwd

( Q, K, V, Sinks, sm_scale, M, Out, # Start_q, Z, H, N_Q_CTX,

src/ollm/gpt_oss_attention.py:18

Method_load_layer_weights

src/ollm/llama.py:48

Method_unload_layer_weights

Replace each loaded attribute with a meta placeholder to free GPU memory.

src/ollm/llama.py:62

Functionexport_nonlayer_weights

scripts/qwen3_next_export.py:29

Functionfile_put_contents

src/ollm/utils.py:4

(self, *args, **kwargs)

src/ollm/voxtral.py:41

(self, *args, **kwargs)

src/ollm/gpt_oss.py:26

(self, hidden_states: torch.Tensor, router_indices=None, routing_weights=None)

src/ollm/gpt_oss.py:33

(self, *args, **kwargs)

src/ollm/gpt_oss.py:116

(self, *args, **kwargs)

src/ollm/qwen3_next.py:224

( self, input_ids: Optional[torch.LongTensor] = None, attention_mask: Optional[torch.Tensor] = None, p

src/ollm/qwen3_next.py:240

(self, x, inputs_embeds, position_ids, position_embeddings)

src/ollm/qwen3_next.py:357

src/ollm/qwen3_next.py:384

(self, *args, **kwargs)

src/ollm/llama.py:96

( self, input_ids: Optional[torch.LongTensor] = None, attention_mask: Optional[torch.Tensor] = None, p

src/ollm/llama.py:111

(ctx, q, k, v, sinks, sm_scale, bandwidth, start_q)

src/ollm/gpt_oss_attention.py:105

(self, *args, **kwargs)

src/ollm/gemma3.py:59

src/ollm/gemma3.py:75

Methodforward_default

(self, hidden_states: torch.Tensor, router_indices=None, routing_weights=None)

src/ollm/gpt_oss.py:59

Methodforward_stacked

(self, hidden_states: torch.Tensor)

src/ollm/qwen3_next.py:111

src/ollm/voxtral.py:55

src/ollm/gpt_oss.py:272

src/ollm/qwen3_next.py:323

src/ollm/llama.py:204

src/ollm/gemma3.py:94

Functiongenerate_manifest

scripts/qwen3_next_export.py:7

Methodget_tensor

(self, name: str)

src/ollm/gds_loader.py:235

(self, name: str)

src/ollm/gds_loader.py:73

Functioninference_audio

scripts/test.py:42

src/ollm/gds_loader.py:232

Functionmy_eager_attention_forward

( module: nn.Module, query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, attention_mask: Optiona

src/ollm/gpt_oss.py:205

Methodoffload_layers_to_cpu

(self, layers_num=2)

src/ollm/voxtral.py:59

Methodoffload_layers_to_cpu

(self, layers_num=2)

src/ollm/gpt_oss.py:276

Methodoffload_layers_to_cpu

(self, layers_num=2)

src/ollm/qwen3_next.py:327

Methodoffload_layers_to_cpu

(self, layers_num=2)

src/ollm/llama.py:189

Methodoffload_layers_to_cpu

(self, layers_num=2)

src/ollm/gemma3.py:98

Methodoffload_layers_to_cpu1

(self, layers_num=2)

src/ollm/llama.py:208

Functiononline_chunked_grouped_attention_rope_no_mask

Online chunked grouped multi-head attention WITHOUT masking (inference-only). - q: (B, Hq, Lq, D) (RoPE already applied) - k: (B, Hkv, Lk

src/ollm/attention.py:5

Methodpreload_layer_safetensors

src/ollm/gds_loader.py:307

Methodpreload_layer_safetensors

src/ollm/gds_loader.py:331

Functionremove_layers_weights

src/ollm/utils.py:78

Methodreorder_cache

(self, beam_idx: torch.LongTensor)

src/ollm/qwen3_next.py:31

Functionsafetensors_vs_ptfiles

scripts/scripts.py:1

Methodstack_params

src/ollm/qwen3_next.py:167

Functiontensor_size_gb

(t: torch.Tensor)

src/ollm/utils.py:13

Functiontest_eq

(batch_size, num_queries, num_keys, num_key_value_heads, num_key_value_groups, head_dim, sm_scale, sliding_win

src/ollm/gpt_oss_attention.py:221

( self, key_states: torch.Tensor, value_states: torch.Tensor, layer_idx: int, cache_kwargs: Optional

src/ollm/qwen3_next.py:34

( self, key_states: torch.Tensor, value_states: torch.Tensor, layer_idx: int, cache_kwargs: Optional

src/ollm/kvcache.py:64

← previous101–149 of 149, ranked by callers