MCPcopy Create free account

hub / github.com/OpenSparseLLMs/Linear-MoE / functions

Functions1,498 in github.com/OpenSparseLLMs/Linear-MoE

⨍ Functions 1,498◇ Types & classes 227
↓ 1 callersMethodgating
Forward pass of the router gate. Args: input (torch.Tensor): Input tensor. Returns: torch.Tensor: Logits ten
linear_moe/model/mixtral/moe/router.py:56
↓ 1 callersMethodgating
Forward pass of the router gate. Args: input (torch.Tensor): Input tensor. Returns: torch.Tensor: Logits ten
linear_moe/model/qwen2/moe/router.py:141
↓ 1 callersMethodgating
Forward pass of the router gate. Args: input (torch.Tensor): Input tensor. Returns: torch.Tensor: Logits ten
linear_moe/model/deepseek_v2/moe/router_old.py:66
↓ 1 callersMethodgating
Forward pass of the router gate. Args: input (torch.Tensor): Input tensor. Returns: torch.Tensor: Logits ten
linear_moe/model/deepseek_v2/moe/router.py:141
↓ 1 callersFunctiongelu_bwd
(g, x)
linear_moe/model/common_modules/activations.py:246
↓ 1 callersFunctiongelu_fwd
(x)
linear_moe/model/common_modules/activations.py:238
↓ 1 callersFunctiongenerate
Given prompts and input parameters, run inference and return the generated tokens, lengths, and output log probabilities. Args:
linear_moe/generation/api.py:114
↓ 1 callersFunctiongenerate_tokens_probs_and_return_on_first_stage
Main token generation function. Args: model: no interleaving is supported. tokens: prompt tokens extended to be of size [b,
linear_moe/generation/generation.py:120
↓ 1 callersFunctionget_batch
Generate a batch.
examples/linear_moe_mixtral/pretrain_mcore_mistral.py:171
↓ 1 callersFunctionget_batch
Generate a batch.
examples/linear_moe_deepseek_v2/pretrain_deepseek.py:165
↓ 1 callersFunctionget_batch
Generate a batch.
examples/linear_moe_qwen2/evaluate_mcore_qwen.py:86
↓ 1 callersFunctionget_batch
Generate a batch.
examples/linear_moe_qwen2/pretrain_qwen.py:157
↓ 1 callersFunctionget_batch
Generate a batch.
examples/linear_llama3/pretrain_llama.py:160
↓ 1 callersFunctionget_bias_dropout_add
(training)
linear_moe/model/llama3/transformer_legacy.py:841
↓ 1 callersFunctionget_file_name
(args, file_id)
toolkits/pretrain_data_preprocessing/preprocess_data_megatron.py:259
↓ 1 callersMethodget_images
(self, return_pil=False)
linear_moe/data/llava/conversation.py:123
↓ 1 callersFunctionget_language_model
Build language model and return along with the key to save.
linear_moe/model/llama3/language_model.py:50
↓ 1 callersFunctionget_model_provider
Based on evaluation metric set the parallel-output flag and return the model provider.
examples/linear_moe_qwen2/evaluate_huggingface_qwen.py:38
↓ 1 callersFunctionget_model_provider
()
examples/linear_moe_qwen2/evaluate_mcore_qwen.py:49
↓ 1 callersFunctionget_num_layers_to_build
(config: TransformerConfig)
linear_moe/model/mixtral/transformer_block.py:42
↓ 1 callersFunctionget_num_layers_to_build
(config: TransformerConfig)
linear_moe/model/qwen2/transformer_block.py:42
↓ 1 callersFunctionget_num_layers_to_build
(config: TransformerConfig)
linear_moe/model/deepseek_v2/transformer_block.py:43
↓ 1 callersMethodget_prompt
(self)
linear_moe/data/llava/conversation.py:43
↓ 1 callersMethodget_query_key_value_tensors
This method needs to be implemented based on whether the derived class is "self-attn" or "cross-attn".
linear_moe/model/mixtral/transformer/attention.py:243
↓ 1 callersMethodget_query_key_value_tensors
This method needs to be implemented based on whether the derived class is "self-attn" or "cross-attn".
linear_moe/model/qwen2/transformer/attention.py:243
↓ 1 callersMethodget_query_key_value_tensors
This method needs to be implemented based on whether the derived class is "self-attn" or "cross-attn".
linear_moe/model/deepseek_v2/transformer/attention.py:171
↓ 1 callersMethodget_query_key_value_tensors
This method needs to be implemented based on whether the derived class is "self-attn" or "cross-attn".
linear_moe/model/llama3/transformer/attention.py:247
↓ 1 callersFunctionget_router_linear_layer
(config)
linear_moe/model/llama3/transformer_legacy.py:190
↓ 1 callersFunctionget_tokenize_len
(prompts)
linear_moe/data/llava/mm_pretrain_dataset.py:154
↓ 1 callersMethodget_vocab
Returns vocab as a dict
linear_moe/tokenizer/tokenization_yi.py:100
↓ 1 callersMethodgpt_convert_example_to_feature
Convert a single sample containing a prompt-answer pair into a format suitable for GPT training.
linear_moe/data/bloom.py:87
↓ 1 callersMethodgpt_convert_example_to_feature
Convert a single sample containing input_id, label and loss_mask into a format suitable for GPT training.
linear_moe/data/llama.py:212
↓ 1 callersMethodgpt_convert_example_to_feature
(self, sample)
linear_moe/data/llama.py:310
↓ 1 callersMethodgpt_convert_example_to_feature
(self, sample, tokenizer)
linear_moe/data/glm.py:189
↓ 1 callersMethodgpt_convert_example_to_feature
(self, sample, tokenizer)
linear_moe/data/glm.py:286
↓ 1 callersMethodgpt_convert_example_to_feature
(self, sample)
linear_moe/data/starcoder.py:137
↓ 1 callersFunctiongroup_norm_fn
( x, weight, bias, residual=None, eps=1e-6, prenorm=False, residual_in_fp32=False,
linear_moe/model/common_modules/layernorm.py:491
↓ 1 callersMethodinit_weights_
Initialize trainable map as identity
linear_moe/model/common_modules/feature_map.py:54
↓ 1 callersMethodinitializer
(self)
toolkits/pretrain_data_preprocessing/preprocess_data.py:37
↓ 1 callersFunctioninsert_separator
(X, sep)
linear_moe/data/llava/mm_utils.py:58
↓ 1 callersFunctionis_power_of_2
(n)
linear_moe/model/common_modules/feature_map.py:34
↓ 1 callersMethodjload
Load a .json file into a dictionary.
linear_moe/data/starcoder.py:73
↓ 1 callersFunctionlasp2_backward
(q, k, v, do, kv, dkv, DKV, BLOCK=128, CBLOCK=64)
linear_moe/sequence_modeling/lasp2/lasp2_with_mask_triton_op.py:871
↓ 1 callersFunctionlasp2_backward
(q, k, v, do, kv, dkv, DKV, BLOCK=128, CBLOCK=64)
linear_moe/sequence_modeling/lasp2/lasp2_without_mask_triton_op.py:612
↓ 1 callersFunctionlasp2_forward
(q, k, v, kv, KV, BLOCK=128, CBLOCK=64)
linear_moe/sequence_modeling/lasp2/lasp2_with_mask_triton_op.py:763
↓ 1 callersFunctionlasp2_forward
(q, k, v, kv, KV, BLOCK=128, CBLOCK=64)
linear_moe/sequence_modeling/lasp2/lasp2_without_mask_triton_op.py:519
↓ 1 callersFunctionload_megatron_model
(args)
toolkits/model_checkpoints_convertor/deepseek/hf2mcore_deepseek_v2_moe.py:58
↓ 1 callersFunctionload_megatron_model
(args, model)
toolkits/model_checkpoints_convertor/llama/hf2mcore.py:221
↓ 1 callersFunctionload_megatron_model
(args, model)
toolkits/model_checkpoints_convertor/mistral/hf2mcore.py:179
↓ 1 callersFunctionload_megatron_model
(args)
toolkits/model_checkpoints_convertor/qwen/hf2mcore_qwen1.5_moe.py:56
↓ 1 callersFunctionload_megatron_model
(args)
toolkits/model_checkpoints_convertor/qwen/hf2mcore_qwen2_dense_and_moe_gqa.py:65
↓ 1 callersFunctionload_megatron_model
(args)
toolkits/model_checkpoints_convertor/qwen/hf2mcore_qwen1.5_dense_mha.py:49
↓ 1 callersFunctionload_megatron_model
(args, model)
toolkits/model_checkpoints_convertor/qwen/hf2megablocks_qwen1.5.py:214
↓ 1 callersFunctionmain
()
toolkits/pretrain_data_preprocessing/preprocess_wudao2.py:46
↓ 1 callersFunctionmain
()
toolkits/pretrain_data_preprocessing/clean_raw_text.py:48
↓ 1 callersFunctionmain
()
toolkits/pretrain_data_preprocessing/preprocess_data_megatron.py:278
↓ 1 callersFunctionmain
()
toolkits/pretrain_data_preprocessing/preprocess_data.py:181
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/falcon40b/checkpoint_reshaping_and_interoperability.py:819
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/deepseek/hf2mcore_deepseek_v2_moe.py:634
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/llama/hf2mcore_70b.py:798
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/llama/hf2megatron.py:1101
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/llama/hf2mcore.py:734
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/mistral/hf2megatron.py:1014
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/mistral/hf2mcore.py:530
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/mistral/hf2mcore_mixtral.py:923
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/baichuan/hf2te.py:524
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/baichuan/checkpoint_reshaping_and_interoperability.py:942
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/glm/checkpoint_reshaping_and_interoperability.py:569
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/falcon/checkpoint_reshaping_and_interoperability.py:843
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/glm130b/checkpoint_reshaping_and_interoperability.py:526
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/starcoder/checkpoint_reshaping_and_interoperability.py:827
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/chatglm/checkpoint_reshaping_and_interoperability.py:583
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/yi/checkpoint_reshaping_and_interoperability.py:669
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/galactica/checkpoint_reshaping_and_interoperability.py:653
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/qwen/hf2megatron_qwen1.0.py:995
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/qwen/hf2mcore_qwen1.5_moe.py:515
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/qwen/hf2mcore_qwen1.5_dense_mha_to_moe.py:230
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/qwen/hf2mcore_qwen2_dense_and_moe_gqa.py:884
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/qwen/hf2mcore_qwen1.5_dense_mha.py:296
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/qwen/hf2mcore_qwen1.5_dense_gqa.py:828
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/qwen/hf2megablocks_qwen1.5.py:613
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/qwen/hf2megatron_qwen1.5.py:1113
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/bloom/checkpoint_reshaping_and_interoperability.py:818
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/bloom/reward_model_to_megatron.py:817
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/bloom/deepspeed_to_megatron_ori.py:163
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/bloom/deepspeed_to_megatron.py:163
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/baichuan2/hf2te.py:496
↓ 1 callersFunctionmain
()
toolkits/model_checkpoints_convertor/baichuan2/checkpoint_reshaping_and_interoperability.py:916
↓ 1 callersFunctionmain
Main program.
examples/linear_moe_qwen2/evaluate_huggingface_qwen.py:104
↓ 1 callersMethodmask_text
(self, text)
linear_moe/data/glm.py:172
↓ 1 callersFunctionmerge_transformers_sharded_states
Merge sharded checkpoints from transformers into a single checkpoint. Args: path (str): the path to the sharded checkpoints n
toolkits/model_checkpoints_convertor/glm/checkpoint_reshaping_and_interoperability.py:219
↓ 1 callersFunctionmerge_transformers_sharded_states
Merge sharded checkpoints from transformers into a single checkpoint. Args: path (str): the path to the sharded checkpoints n
toolkits/model_checkpoints_convertor/glm130b/checkpoint_reshaping_and_interoperability.py:219
↓ 1 callersFunctionmerge_transformers_sharded_states
Merge sharded checkpoints from transformers into a single checkpoint. Args: path (str): the path to the sharded checkpoints n
toolkits/model_checkpoints_convertor/starcoder/checkpoint_reshaping_and_interoperability.py:270
↓ 1 callersFunctionmerge_transformers_sharded_states
Merge sharded checkpoints from transformers into a single checkpoint. Args: path (str): the path to the sharded checkpoints n
toolkits/model_checkpoints_convertor/chatglm/checkpoint_reshaping_and_interoperability.py:249
↓ 1 callersFunctionmerge_transformers_sharded_states
Merge sharded checkpoints from transformers into a single checkpoint. Args: path (str): the path to the sharded checkpoints n
toolkits/model_checkpoints_convertor/bloom/checkpoint_reshaping_and_interoperability.py:248
↓ 1 callersFunctionmerge_transformers_sharded_states
Merge sharded checkpoints from transformers into a single checkpoint. Args: path (str): the path to the sharded checkpoints n
toolkits/model_checkpoints_convertor/bloom/reward_model_to_megatron.py:248
↓ 1 callersFunctionmerge_transformers_sharded_states
Merge sharded checkpoints from transformers into a single checkpoint. Args: path (str): the path to the sharded checkpoints n
toolkits/model_checkpoints_convertor/baichuan2/hf2te.py:164
↓ 1 callersFunctionmerge_transformers_sharded_states_13b
Merge sharded checkpoints from transformers into a single checkpoint. Args: path (str): the path to the sharded checkpoints n
toolkits/model_checkpoints_convertor/llama/hf2megatron.py:308
↓ 1 callersFunctionmerge_transformers_sharded_states_13b
Merge sharded checkpoints from transformers into a single checkpoint. Args: path (str): the path to the sharded checkpoints n
toolkits/model_checkpoints_convertor/baichuan/hf2te.py:180
↓ 1 callersFunctionmerge_transformers_sharded_states_13b
Merge sharded checkpoints from transformers into a single checkpoint. Args: path (str): the path to the sharded checkpoints n
toolkits/model_checkpoints_convertor/baichuan/checkpoint_reshaping_and_interoperability.py:292
← previousnext →501–600 of 1,498, ranked by callers