MCPcopy Create free account
hub / github.com/OpenSparseLLMs/Linear-MoE / forward

Method forward

linear_moe/model/common_modules/layernorm.py:656–665  ·  view source on GitHub ↗
(self, x, residual=None, prenorm=False, residual_in_fp32=False)

Source from the content-addressed store, hash-verified

654 return s
655
656 def forward(self, x, residual=None, prenorm=False, residual_in_fp32=False):
657 return rms_norm_fn(
658 x,
659 self.weight,
660 self.bias,
661 residual=residual,
662 eps=self.eps,
663 prenorm=prenorm,
664 residual_in_fp32=residual_in_fp32,
665 )
666
667
668class LayerNormLinearFn(torch.autograd.Function):

Callers

nothing calls this directly

Calls 1

rms_norm_fnFunction · 0.85

Tested by

no test coverage detected