Return storage pointers for tests and benchmark transport verification.

Estimate floating-point operations for a single logp forward pass. log_softmax ~= 5 * B * S * V FLOPs (subtract max, exp, s

Extract scalar reward groups from rollout payloads when they are present.

Compute `a @ b` and return the input dtype.

Apply RoPE in input dtype. Cos/sin always computed in fp32.

Build a sampler config from the executor's loose model_config dictionary.

Returns the optimal data type for the current hardware. AMD ROCm typically yields better performance with bfloat16 in RL workloads.

Per-sequence reward normalization, computed by the Triton group kernel.

Allocate paged KV tensors and a dense block table for the scoring batch.

Whether a linear_logp call describes a vocab-parallel weight shard.

Count tensors and bytes in nested tuples/lists/mappings/dataclass-like outputs.

Core softmax attention: cast to ``compute_dtype``, score, mask, softmax, weighted-sum over V, cast out. ``strict_fp32`` disables autocast/TF32

Compute cos/sin tables in fp32 from positions and theta. Args: x: [..., D] — only x.shape[-1] (head_dim) is used. pos

Append the step's new K/V to the cache along the seq axis (dim=2). Layout is [B, Hkv, S, D], so the sequence axis is dim=2. Pure (no

Inject a stub ``flashinfer.sampling`` so the FlashInfer code path runs on CPU. ``top_p_sampling_from_probs`` records the probabilities it receive

Core matmul: cast to ``compute_dtype``, project, optionally add bias, cast out. When ``strict_fp32`` is set, the matmul runs with autocast di

Implementation for one-time logging using LRU cache.

Set garbage at masked positions' logits; the loss must ignore them.

Mirror of examples.grpo_single_gpu.make_group_advantages normalization.

Return a stable identifier for a CUDA device. torch.cuda.get_device_properties(...).uuid only exists on PyTorch >= 2.6. This project sup