MCPcopy Create free account

hub / github.com/cwxndl/LLM / functions

Functions215 in github.com/cwxndl/LLM

⨍ Functions 215◇ Types & classes 43↳ Endpoints 3
↓ 9 callersMethod__init__
(self, config=ndlconfig,hidden_size = None, intermediate_size = None)
model/modeling_moe.py:167
↓ 7 callersMethod__init__
(self, config:ndlconfig)
model/modeling_ndl.py:170
↓ 6 callersFunctioncall_with_stream
(prompt,url)
chat-bot/app_multi_conversation.py:42
↓ 5 callersFunctionsplit_txt_cropus_to_chunk_data
( texts: list, batch_size: int = 512**2, max_len: int = 512, window_size: int = 2 )
utils/data_preprocess.py:21
↓ 4 callersMethodbm_25
(self)
chat-bot/doc_retrieve.py:24
↓ 4 callersFunctioncall_with_stream
(prompt,url)
chatbot3.0/webui.py:79
↓ 4 callersFunctionrepeat_kv
(hidden_states: torch.Tensor, n_rep: int)
model/modeling_ndl.py:186
↓ 4 callersFunctionrepeat_kv
(hidden_states: torch.Tensor, n_rep: int)
model/modeling_moe.py:326
↓ 3 callersFunctionprocess_pdf
(pdf_path)
chat-bot/file_preprocess.py:14
↓ 2 callersFunctionapply_rope
输入参数: q (`torch.Tensor`): q向量. k (`torch.Tensor`): k向量. cos (`torch.Tensor`): 旋转编码的余弦值. sin (`torch.Tensor`): 旋转编码的正弦值. un
model/modeling_ndl.py:147
↓ 2 callersFunctionapply_rope
输入参数: q (`torch.Tensor`): q向量. k (`torch.Tensor`): k向量. cos (`torch.Tensor`): 旋转编码的余弦值. sin (`torch.Tensor`): 旋转编码的正弦值. un
model/modeling_moe.py:144
↓ 2 callersMethodbm_25
(self)
chatbot3.0/doc_retrieve.py:45
↓ 2 callersMethodforward
(self,attention_out)
model/modeling_ndl.py:179
↓ 2 callersMethodforward
(self,attention_out)
model/modeling_moe.py:176
↓ 2 callersFunctionimage2text_call
(image_name,prompt)
chat-bot/app_multi_conversation.py:69
↓ 2 callersFunctionimage2text_call
(image_name,prompt)
chatbot3.0/webui.py:104
↓ 2 callersFunctionprocess_md
(md_path)
chat-bot/file_preprocess.py:38
↓ 2 callersFunctionprocess_none
(s: str)
utils/data_preprocess.py:44
↓ 2 callersFunctionprocess_pptx
(pptx_path)
chatbot3.0/file_preprocess.py:177
↓ 2 callersFunctionrotate_half
(x)
model/modeling_ndl.py:124
↓ 2 callersFunctionrotate_half
(x)
model/modeling_moe.py:121
↓ 1 callersFunction_gc
()
chat-bot/app_ndl.py:59
↓ 1 callersFunction_gc
()
chat-bot/app_multi_conversation.py:236
↓ 1 callersFunction_gc
()
chat-bot/app_qwen.py:59
↓ 1 callersFunction_gc
()
chatbot3.0/webui.py:321
↓ 1 callersMethod_rms_norm
(self, input)
model/modeling_ndl.py:64
↓ 1 callersMethod_rms_norm
(self, input)
model/modeling_moe.py:61
↓ 1 callersFunctioncache_data
(data, file_name)
finetune.py:23
↓ 1 callersFunctioncache_data
(data, file_name)
finetune_qwen.py:23
↓ 1 callersFunctioncalculate_file_hash
(file_path)
chat-bot/app_ndl.py:14
↓ 1 callersFunctioncalculate_file_hash
(file_path)
chat-bot/app_multi_conversation.py:94
↓ 1 callersFunctioncalculate_file_hash
(file_path)
chat-bot/app_qwen.py:14
↓ 1 callersFunctioncalculate_file_hash
(file_path)
chatbot3.0/webui.py:129
↓ 1 callersFunctioncall_city_weather
(city)
chatbot3.0/webui.py:54
↓ 1 callersFunctioncall_image_generation
(prompt)
chatbot3.0/webui.py:68
↓ 1 callersFunctioncall_llm
(prompt, url="http://127.0.0.1:6666/chat")
chat-bot/app_ndl.py:21
↓ 1 callersFunctioncall_llm
(prompt, url="http://127.0.0.1:6666/chat")
chat-bot/app_qwen.py:21
↓ 1 callersFunctioncall_rerank
(prompt,url = "http://127.0.0.1:9999/rerank")
chatbot3.0/doc_retrieve.py:9
↓ 1 callersFunctionchat
(stream: bool=True)
cli_qwen_demo.py:34
↓ 1 callersFunctionchat
(stream: bool=True)
cli_demo.py:47
↓ 1 callersFunctioncheck_dir_exits
(dir: str)
tokenizer_train/1_train_tokenizer.py:17
↓ 1 callersFunctionclassify_task
(prompt,url = 'qwen-turbo')
chat-bot/app_multi_conversation.py:23
↓ 1 callersFunctionclassify_task
(prompt,url = 'qwen-turbo')
chatbot3.0/webui.py:37
↓ 1 callersMethodextract_text
(shape)
chatbot3.0/file_preprocess.py:140
↓ 1 callersFunctiongen_sky
(input_folder, output_folder)
utils/data_preprocess.py:113
↓ 1 callersMethodgenerate
( self, inputs: Optional[torch.Tensor] = None, generation_config: Optional[Generati
model/modeling_ndl.py:799
↓ 1 callersMethodget_docs_prompt
(self,end_top_k=4)
chatbot3.0/doc_retrieve.py:62
↓ 1 callersFunctionget_maped_dataset
(files)
ex_pretrain.py:159
↓ 1 callersFunctionget_maped_dataset
(files)
pretrain.py:160
↓ 1 callersFunctionget_maped_dataset
(files)
moe_pretrain.py:70
↓ 1 callersFunctionget_ocr
(use_cuda: bool = True)
chatbot3.0/file_preprocess.py:22
↓ 1 callersMethodget_rope
(self)
model/modeling_ndl.py:224
↓ 1 callersMethodget_rope
(self)
model/modeling_moe.py:364
↓ 1 callersFunctionget_training_corpus
从文本文件中生成训练语料库的生成器函数。 Args: buffer_size (int): 每次生成的文本块数量。 chunk_len (int): 每个文本块的最大长度。 Retur
tokenizer_train/1_train_tokenizer.py:26
↓ 1 callersFunctionload_cached_data
(file_name)
finetune.py:26
↓ 1 callersFunctionload_cached_data
(file_name)
finetune_qwen.py:26
↓ 1 callersFunctionmain
()
ex_pretrain.py:240
↓ 1 callersFunctionmain
()
finetune.py:182
↓ 1 callersFunctionmain
()
pretrain.py:239
↓ 1 callersFunctionmain
()
finetune_qwen.py:182
↓ 1 callersFunctionmain
()
moe_pretrain.py:157
↓ 1 callersFunctionmain
()
chat-bot/app_ndl.py:72
↓ 1 callersFunctionmain
()
chat-bot/app_multi_conversation.py:256
↓ 1 callersFunctionmain
()
chat-bot/app_qwen.py:72
↓ 1 callersFunctionmain
()
chatbot3.0/webui.py:341
↓ 1 callersFunctionmain
()
my_lora/finetune.py:167
↓ 1 callersFunctionmake_supervised_data_module
Make dataset and collator for supervised fine-tuning.
finetune.py:163
↓ 1 callersFunctionmake_supervised_data_module
Make dataset and collator for supervised fine-tuning.
finetune_qwen.py:163
↓ 1 callersFunctionmake_supervised_data_module
Make dataset and collator for supervised fine-tuning.
my_lora/finetune.py:149
↓ 1 callersMethodmoe_infer
(self, x, flat_expert_indices, flat_expert_weights)
model/modeling_moe.py:308
↓ 1 callersFunctionpreprocess
( sources, tokenizer, max_len: int, system_message: str = "You are a helpful assistant."
finetune.py:86
↓ 1 callersFunctionpreprocess
( sources, tokenizer, max_len: int, system_message: str = "You are a helpful assistant."
finetune_qwen.py:87
↓ 1 callersFunctionpreprocess
( sources, tokenizer, max_len: int, system_message: str = "You are a helpful assistant." )
my_lora/finetune.py:79
↓ 1 callersFunctionprocess_md
(md_path)
chatbot3.0/file_preprocess.py:120
↓ 1 callersFunctionprocess_pdf
(pdf_path)
chatbot3.0/file_preprocess.py:60
↓ 1 callersFunctionprocess_txt
(txt_path)
chat-bot/file_preprocess.py:23
↓ 1 callersFunctionprocess_txt
(txt_path)
chatbot3.0/file_preprocess.py:105
↓ 1 callersFunctionprocess_url
(url)
chatbot3.0/file_preprocess.py:31
↓ 1 callersFunctionprocess_word
(word_path)
chat-bot/file_preprocess.py:27
↓ 1 callersFunctionprocess_word
(word_path)
chatbot3.0/file_preprocess.py:109
↓ 1 callersMethodrerank_docs
(self,end_top_k,first_selected_docs)
chatbot3.0/doc_retrieve.py:52
↓ 1 callersMethodreset_parameters
(self)
model/modeling_moe.py:197
↓ 1 callersFunctionsave_text
(content,file_name)
chat-bot/app_multi_conversation.py:116
↓ 1 callersFunctionsave_text
(content,file_name)
chatbot3.0/webui.py:151
↓ 1 callersFunctionsft_zh
()
utils/sft_data.py:10
↓ 1 callersMethodtext_split
(self)
chat-bot/doc_retrieve.py:16
↓ 1 callersMethodtext_split
(self)
chatbot3.0/doc_retrieve.py:37
↓ 1 callersFunctiontoken_to_id
(samples: dict)
ex_pretrain.py:142
↓ 1 callersFunctiontoken_to_id
(samples: dict)
pretrain.py:143
↓ 1 callersFunctiontoken_to_id
(samples: dict)
moe_pretrain.py:51
↓ 1 callersFunctiontorch_gc
()
chat-bot/NDLmodel_api.py:10
↓ 1 callersFunctiontorch_gc
()
chat-bot/qwen2_api.py:13
↓ 1 callersFunctiontorch_gc
()
chatbot3.0/rerank_api.py:14
↓ 1 callersFunctiontrain_tokenizer
(cropus_file: str, max_train_line: int=None, vocab_size: int=40960,token_type: str='char')
tokenizer_train/1_train_tokenizer.py:21
Method__getitem__
(self, i)
finetune.py:157
Method__getitem__
(self, i)
finetune_qwen.py:157
Method__getitem__
(self, i)
my_lora/finetune.py:143
Method__init__
(self, raw_data, tokenizer, max_len: int,cache_file)
finetune.py:137
Method__init__
(self, raw_data, tokenizer, max_len: int,cache_file)
finetune_qwen.py:138
Method__init__
(self, query, top_k, chunk_size, chunk
chat-bot/doc_retrieve.py:5
next →1–100 of 215, ranked by callers