Functions — github.com/DLLXW/baby-llama2-chinese

CodeHub MCPcopy Create free account

hub / github.com/DLLXW/baby-llama2-chinese / functions

Functions108 in github.com/DLLXW/baby-llama2-chinese

⨍ Functions 108◇ Types & classes 14

Functionsave_model_config

将模型配置写入到json文件, 输入模型保存的目录及文件名

data_clean/functions.py:389

Methodsave_vocabulary

Save the vocabulary and special tokens file to a directory. Args: save_directory (`str`): The directory

chatglm_tokenizer/tokenization_chatglm.py:130

Methodserialize

Functionsft_to_pretrain

data_process.py:39

Functionshuffle_parquet_dataset

打乱一个parquet文件数据集

data_clean/clear.py:244

Methodunk_token

chatglm_tokenizer/tokenization_chatglm.py:87

Functionvalid_epoch

(self, message: str, std_out: bool=True, save_to_file: bool=False)

data_clean/logger.py:86

← previous101–108 of 108, ranked by callers