Muon - MomentUm Orthogonalized by Newton-schulz https://kellerjordan.github.io/posts/muon/ Muon internally runs standard SGD-momentum,

Distributed Muon variant that can be used for all parameters in the network, since it runs an internal AdamW for the parameters that are not

Muon variant for usage in non-distributed settings.

Non-distributed variant of MuonWithAuxAdam.