Build a launchable kernel.

Check correctness with random shapes (similar to flux)

Params: initial_state: Return: final_state: fp32 [N, H, K, V] if output_final_state is True, otherwise None

varlen mode k: [1, T, H, K] v: [1, T, H, V] beta: [1, T, H] g: [1, T, H] cu_seqlens: [B + 1] chunk_indices: [NT * 2] wher

Color latency - always green

Color memory - always blue

Compile CUDA code to PTX or CUBIN using nvcc. Parameters ---------- code : str The CUDA source code. kernel_name : s

Compute dispatch layout: all-gather splits, cumsum -> scatter indices + send mask. Parameters ---------- topk_indice

Compute stable within-expert token offsets and expert counts. Parameters ---------- topk_indices : torch.Tensor[int3

triton implementation of cooperative_group::this_grid().sync() WARNING: use with care. better launch triton with launch_cooperative_grid=True to

Make source path a copy of the target path. if both source/target under base_dir, create link with relative path. why it's tricky for that?

Cosine function (approximate).

Create a 2D TMA descriptor. Parameters ---------- tensor : torch.Tensor Input tensor gme

for num_reduction_sms: tunable param, 16 sms are enough for H800 For H800, we overlap local reduce and inter-node p2p with intra-

Dont call this in main(), avoid call mxshmem_free(ptr) after mxshmem_finalize

Wrap a pass function with debug output.