MCPcopy Create free account
hub / github.com/ckd0817/LLM-Interview-Code / KL_divergence

Function KL_divergence

loss/EntropyLoss.py:103–134  ·  view source on GitHub ↗

KL 散度(Kullback-Leibler Divergence) 衡量两个概率分布 P 和 Q 之间的差异。 公式: D_KL(P || Q) = sum(P(x) * (log P(x) - log Q(x))) 常用于知识蒸馏、DPO 训练等场景。 Args: p_logits: 分布 P 的 logits [batch_size, num_classes] q_logits: 分布 Q 的 logits [batch_size, num_classes] Returns: kl_

(p_logits, q_logits)

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Callers

nothing calls this directly

Calls 2

log_softmaxFunction · 0.85
softmaxFunction · 0.85

Tested by

no test coverage detected