Forward-style grouped GEMM: a[M,KC] x b[E,KC,NO] -> out[M,NO], grouped over M.

Per-parameter selective update for offloaded partitions (CPU or NVMe). NVMe partitions are swapped in, updated, and swapped back out one at a

Check if module stores expert weights as 3D parameter tensors (transformers 5.0.0+). Returns True if the module has a parameter named preset.expe

Returns whether or not all parameters have been initialized and transformed by the model. Once this returns True, all the `ParameterB

Merge-base of ``base`` and ``HEAD``, or "" if it can't be determined. An empty result (e.g. shallow clone that doesn't reach the fork point)

Dotted module name for a file under a package root, or None. e.g. deepspeed/runtime/engine.py -> deepspeed.runtime.engine tests/

Parse every file once; return (deps_by_file, raw_imports_by_file).

Returns the prime factorization of positive integer N.

Process gradients for selected columns in FP32 groups Args: param: The parameter to process param_id: ID of

All-reduce the input tensor across model parallel group.

Reduce (sum) a new local result into the existing local results. Subclasses should override this method. For a given ``out_id``, thi

Reference for _group_meta: m_start = exclusive-prefix, m_size = per-group count.

Determine which preset(s) to use for detection.

Resolve the single scale applied by TokenChoiceTopKRouter.

Map each file -> the set of files that import it (directly).

change sign so the last dimension becomes [-odd, +even]

Map every (token, route) pair onto its flat slot in the [e * c] expert buffer. Routes evicted by the capacity limit are parked on one extra scrat

Compute SwiGLU expert MLP via torch._grouped_mm (grouped GEMM). Uses input dtype for casting instead of hardcoded bfloat16. Args: w1

Run zero3_linear_wrap forward+backward, optionally inside autocast.

Toggle DeepCompile runtime state and manage forward hooks accordingly.