Functions — github.com/LLMBook-zh/LLMBook-zh.github.io

CodeHub MCPcopy Create free account

hub / github.com/LLMBook-zh/LLMBook-zh.github.io / functions

Functions49 in github.com/LLMBook-zh/LLMBook-zh.github.io

⨍ Functions 49 ◇ Types & classes 17

↓ 2 callersMethod_forward_rmloss

(self, input_ids, attention_mask, **kargs)

code/8.1 奖励模型训练.py:14

↓ 2 callersMethodencode

(self, examples)

code/6.3 预训练数据类.py:25

↓ 2 callersMethodforward

(self, sent1_idx, attention_mask_1, sent2_idx, attention_mask_2, labels, prompt_ids, lm_attn_mask, response_id

code/8.1 奖励模型训练.py:50

↓ 2 callersFunctionrotate_half

code/5.2 RoPE.py:1

↓ 1 callersMethod_forward_lmloss

(self, prompt_ids, lm_attn_mask, response_ids)

code/8.1 奖励模型训练.py:29

↓ 1 callersFunctiondequantize_func

(x_q, scales, zero_point)

code/9.2 量化示例.py:9

↓ 1 callersMethodencode_src_tgt

(self, s, t, tokenizer)

code/7.2 SFT数据类.py:37

↓ 1 callersFunctionencode_with_bpe

使用字节对编码(BPE)算法对输入文本进行编码。此函数首先提取词元的初始频率，然后迭代合并频率最高的字符对，直到达到指定的合并次数或没有可合并的对为止。参数: texts (list of str): 输入的字符串列表，每个字符串代表一段文本。 num

code/4.4 BPE分词.py:61

↓ 1 callersFunctionextract_frequencies

将输入文本列表中的每个文本转换为带有结束标记'</w>'的单个字符序列，并计算每种序列的频率。此函数使用Counter来累加每种序列的出现次数，这样可以快速得到每个序列在文本中出现的总次数。参数: texts (list of str): 输

code/4.4 BPE分词.py:4

↓ 1 callersFunctionfrequency_of_pairs

从给定的频率字典中计算所有相邻字符对的频率。通过遍历每个词元，查找并统计所有相邻字符对的出现频率。参数: frequencies (Counter): 词元到其频率的映射字典。返回: Counter: 字符对到其频率的映射字典。

code/4.4 BPE分词.py:22

↓ 1 callersFunctionget_data

(split, data_path)

code/8.2 DPO实践.py:36

↓ 1 callersMethodgroup_texts

(self, examples)

code/6.3 预训练数据类.py:30

↓ 1 callersFunctionmerge_vocab

合并词汇表中最频繁的字符对。此函数接受一个字符对和当前的词汇表，将所有包含该字符对的词元中的对应字符合并为一个单一字符，并更新词汇表以反映这一变化。参数: pair (tuple): 要合并的字符对。 vocab (Counter): 当前的

code/4.4 BPE分词.py:40

↓ 1 callersMethodprocess

(self, tokenizer)

code/7.2 SFT数据类.py:48

↓ 1 callersMethodprocess

code/6.3 预训练数据类.py:41

↓ 1 callersFunctionquantize_func

(x, scales, zero_point, n_bits=8)

code/9.2 量化示例.py:4

↓ 1 callersFunctiontrain

code/7.1 SFT实践.py:64

↓ 1 callersFunctiontrain

code/6.2 预训练实践.py:43

↓ 1 callersFunctiontrain

code/8.2 DPO实践.py:53

↓ 1 callersFunctiontrain

code/7.4 LoRA实践.py:32

(self, instances)

code/7.1 SFT实践.py:52

Method__getitem__

code/7.2 SFT数据类.py:33

Method__getitem__

code/6.3 预训练数据类.py:21

code/6.1 LM损失.py:2

code/4.3 隐私过滤.py:5

code/4.2 去重.py:6

(self, config: LlamaConfig)

code/5.5 LLaMA.py:2

code/8.1 奖励模型训练.py:8

(self, args, tokenizer)

code/7.2 SFT数据类.py:22

(self, experts: List[nn.Module], gate: nn.Module, num_experts_per_tok: int)

code/5.4 MoE.py:2

(self, config: LlamaConfig, layer_idx: int)

code/5.6 LLaMALayer.py:2

(self, args, tokenizer)

code/6.3 预训练数据类.py:8

code/4.1 质量过滤.py:4

(self, in_features, out_features, config, bias=True)

code/7.3 LoRA基础.py:4

(self, hidden_size, eps=1e-6)

code/5.1 RMSNorm.py:2

code/7.2 SFT数据类.py:29

code/6.3 预训练数据类.py:17

Functionapply_rotary_pos_emb

(q, k, cos, sin, position_ids)

code/5.2 RoPE.py:7

Functionbuild_alibi_tensor

(attention_mask: torch.Tensor, num_heads: int, dtype: torch.dtype)

code/5.3 ALiBi.py:1

Methodclean_single_text

(self, text: str, repl_text: str = "**MASKED**IDCARD**")

code/4.3 隐私过滤.py:7

Methodclean_single_text

(self, text: str, n: int = 5, thre_sim: float = 0.95)

code/4.2 去重.py:11

Methodfilter_single_text

(self, text: str, accept_lang_list: list)

code/4.1 质量过滤.py:8

Functionforward

( self, input_ids: torch.LongTensor = None, attention_mask: Optional[torch.Tensor] = N

code/5.5 LLaMA.py:18

( self, input_ids: torch.LongTensor = None, attention_mask: Optional[torch.Tensor] = N

code/6.1 LM损失.py:8

(self, inputs: torch.Tensor)

code/5.4 MoE.py:10

( self, hidden_states: torch.Tensor, attention_mask: Optional[torch.Tensor] = None,

code/5.6 LLaMALayer.py:13

code/7.3 LoRA基础.py:22

(self, hidden_states)

code/5.1 RMSNorm.py:7

Functionsplit_prompt_and_responses_hh

code/8.2 DPO实践.py:39