MCPcopy Create free account

hub / github.com/chaoyi-wu/PMC-LLaMA / functions

Functions30 in github.com/chaoyi-wu/PMC-LLaMA

⨍ Functions 30◇ Types & classes 9
↓ 1 callersFunction_tokenize_fn
Tokenize a list of strings.
SFT/train.py:81
↓ 1 callersFunctionconstruct_spedical_tokens_dict
()
SFT/eval/qa_inference.py:43
↓ 1 callersFunctioninference_on_one
(input_str: Sequence[str], model, tokenizer)
SFT/eval/qa_inference.py:87
↓ 1 callersFunctionmain
()
Pretrain/training.py:56
↓ 1 callersFunctionmain
()
Pretrain/Data/tokenize_dataset.py:28
↓ 1 callersFunctionmake_supervised_data_module
Make dataset and collator for supervised fine-tuning.
SFT/train.py:168
↓ 1 callersFunctionparse_args
()
SFT/eval/eval_medqa.py:15
↓ 1 callersFunctionparse_args
()
SFT/eval/qa_inference.py:34
↓ 1 callersFunctionparse_pmc_answers
(result_dir: str, file_names: Sequence[str])
SFT/eval/eval_medqa.py:34
↓ 1 callersFunctionpreprocess
Preprocess the data by tokenizing.
SFT/train.py:105
↓ 1 callersMethodrandom_subsection
(self, arr)
Pretrain/Dataset/Paper_dataset.py:59
↓ 1 callersFunctionread_jsonl
(file_path)
SFT/eval/qa_inference.py:112
↓ 1 callersFunctionread_jsonl
(path)
Pretrain/Data/tokenize_dataset.py:12
↓ 1 callersFunctionsafe_save_model_for_hf_trainer
Collects the state dict and dump to disk.
Pretrain/training.py:28
↓ 1 callersFunctionsentence_make
(sentence,reflect_array,special_tokens_list)
Pretrain/Data/tokenize_dataset.py:18
↓ 1 callersFunctionsmart_tokenizer_and_embedding_resize
Resize tokenizer and embedding. Note: This is the unoptimized version that may make your embedding size not be divisible by 64.
SFT/train.py:58
↓ 1 callersFunctionsmart_tokenizer_and_embedding_resize
Resize tokenizer and embedding. Note: This is the unoptimized version that may make your embedding size not be divisible by 64.
SFT/eval/qa_inference.py:62
↓ 1 callersFunctiontrain
()
SFT/train.py:176
↓ 1 callersFunctiontraversal_files
(dir_path: str, extension: str)
SFT/eval/eval_medqa.py:23
Method__call__
(self, instances: Sequence[Dict])
SFT/train.py:155
Method__getitem__
(self, i)
SFT/train.py:145
Method__getitem__
(self,idx)
Pretrain/Dataset/Paper_dataset.py:52
Method__init__
(self, data_path: str, tokenizer: transformers.PreTrainedTokenizer)
SFT/train.py:123
Method__init__
(self, root_path, dir_name, seq_length = 512, voc_size = 32000,keep_author = True, keep_bib = True)
Pretrain/Dataset/Paper_dataset.py:39
Method__len__
(self)
SFT/train.py:142
Method__len__
(self)
Pretrain/Dataset/Paper_dataset.py:49
Functionprepare_data
(data_list: Sequence[dict], model, tokenizer)
SFT/eval/qa_inference.py:120
Functionread_json
(path)
Pretrain/training.py:24
Functionread_jsonl
(path,eval_num)
Pretrain/Dataset/Paper_dataset.py:22
Functionwrite_json
(x, path)
Pretrain/training.py:20