Runs the kernel using initialized state.

Runs the kernel using initialized state.

Primary run() entry point API that is static allowing users to create and manage their own params. Supplied params struct must be construct by calling

Runs the kernel using initialized state.

Runs the kernel using initialized state.

Runs the kernel using initialized state.

Run warmup, graph capture, replay warmup, and timed graph replays. Timing is read from Nsight Systems' NVTX GPU projected duration. Do not sy

FusedMoE-style timing worker: warmup, graph capture, replay under NVTX.

Run worker.py under nsys profile.

Set a bit within the predicate vector.

Overrides the internal iteration index

Overrides the internal iteration index

Overrides the internal iteration index

Overrides the internal iteration index

Overrides the internal iteration index

Overrides the internal iteration index

Overrides the internal iteration index

Overrides the internal iteration index

Overrides the internal iteration index

Overrides the internal iteration index

Overrides the internal iteration index

Overrides the internal iteration index

Sets the predicate mask, overriding value stored in predicate iterator

Sets the predicate mask, overriding value stored in predicate iterator

Sets the predicate mask, overriding value stored in predicate iterator

Build tensors and return the timed call_fn.

Warp shuffle reduction

Butterfly reduction

Returns the sign bit

Returns the sign bit

Compute dim128 block_logits via OAM GEMM with fused causal mask epilogue. Performs block_logits = FrobScale * (Qflat @ Kflat^T) + V_bias, with

Precompute K_flat and V_bias from paged FP8 KV cache. First stage of the Stem sparse scoring pipeline: - K_flat (BF16): group-summed K vect

Precompute dim128 Q_flat from packed FP8 Q tensor. Computes weighted group-sum of Q tokens using per-token qscale (Q FP8 dequantization scale

Generate sparse block mask via top-k policy denoising. Fuses per-row budget (3-regime k_schedule + linear decay, both keyed on the full promp

Stores a fragment to memory

Stores a fragment on the diagonal of a symmetric kernel to memory

Stores a fragment to memory

Stores a fragment to memory

Stores a fragment to memory

Store a fragment to memory

Store a fragment to memory

Store a fragment to memory

Store a fragment to memory

Store

Swaps the managed objects with *this and another unique_ptr