MCPcopy Create free account
hub / github.com/OpenSparseLLMs/Linear-MoE / rms_norm_fn

Function rms_norm_fn

linear_moe/model/common_modules/layernorm.py:515–533  ·  view source on GitHub ↗
(
    x,
    weight,
    bias,
    residual=None,
    eps=1e-6,
    prenorm=False,
    residual_in_fp32=False
)

Source from the content-addressed store, hash-verified

513
514
515def rms_norm_fn(
516 x,
517 weight,
518 bias,
519 residual=None,
520 eps=1e-6,
521 prenorm=False,
522 residual_in_fp32=False
523):
524 return LayerNormFn.apply(
525 x,
526 weight,
527 bias,
528 residual,
529 eps,
530 prenorm,
531 residual_in_fp32,
532 True
533 )
534
535
536class LayerNorm(nn.Module):

Callers 1

forwardMethod · 0.85

Calls

no outgoing calls

Tested by

no test coverage detected