hub / github.com/ckd0817/LLM-Interview-Code / RMSNorm

Class RMSNorm

normalization/RMSNorm.py:14–68 · view source on GitHub ↗

RMS 归一化模块公式: RMSNorm(x) = x / RMS(x) * gamma 其中 RMS(x) = sqrt(mean(x^2) + eps) 相比 LayerNorm，RMSNorm 不计算均值，计算量更小。 Args: model_dim: 归一化的维度（通常是 model_dim） eps: 数值稳定性常数，防止除零，默认 1e-8

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Callers

nothing calls this directly

Calls

no outgoing calls

Tested by

no test coverage detected