Builder for type-safe kernel launches across CUDA + Metal. Accumulates grid dimensions, block dimensions, and typed kernel arguments. `launch()` pack

Quantize input tensor to FP8 (per-tensor, per-token, per-channel, or per-group). This CustomOp supports both static and dynamic quantization.

One stage of the pre-sample pipeline. Implementations are pure-CPU today; a future GPU-resident bitmask kernel can implement this trait too without ch

Per-request timing collected from a single streaming call.

Supports loading kv-cache scaling factors from FP8 checkpoints.

Linear method for FP8. Supports loading FP8 checkpoints with static weight scale and dynamic/static activation scale. Limitations: 1.

MoE method for FP8. Supports loading FP8 checkpoints with static weight scale and dynamic/static activation scale. Also supports loading

Online version of Fp8LinearMethod which loads a full precision checkpoint and quantizes weights during loading.

MoE method for online FP8 quantization. Supports loading quantized FP16/BF16 model checkpoints with dynamic activation scaling. The weight sca

───────────────────────── deterministic PRNG ─────────────────────────

splitmix64 — reproducible inputs, no rand dependency (PCND: explicit seed).

───────────────────────── deterministic PRNG (splitmix64) ─────────────────

───────────────────────── deterministic PRNG ─────────────────────────

───────────────────────── deterministic PRNG ─────────────────────────

───────────────────────── deterministic PRNG ───────────────────────── splitmix64 — reproducible test inputs without a `rand` dependency (PCND: seed i

───────────────────────── deterministic PRNG ───────────────────────── splitmix64 — reproducible test inputs without a `rand` dependency (PCND: seed i

───────────────────────── deterministic PRNG ─────────────────────────

Activation function backend trait.

An in-flight sequence participating in batched decode.

Per-sequence timing for decode urgency decisions.

Pipeline stage that records Shannon entropy of the masked logit distribution to a JSONL file. Diagnostic-only — never mutates logits.

Per-sequence adaptive sampling state machine.

Heap buffer aligned to [`O_DIRECT_ALIGN`]. `Send` so it can cross a channel from the reader thread to the copier thread.

Analyze which rules can match the empty string. Returns the sorted list of nullable rule ids.

State machine kept alive for the lifetime of one /v1/messages streaming response. Each incoming OpenAI chunk JSON is fed in via `process_openai_chunk`

Shared application state accessible from all HTTP handlers.

Apple Metal compilation target: `.metal` → `.metallib` via the two-step `xcrun -sdk macosx metal -c` (→ AIR) then `xcrun -sdk macosx metallib` (→ meta

Per-arg metadata: which slot of `storage` it lives at, and how many native bytes it occupies. Buffer args set `is_buffer = true` and ignore `byte_len`

Production GPU backend wrapping AtlasRegistry + raw CUDA driver API. Initialized once via [`AtlasCudaBackend::new`], which loads all PTX modules from

Cached CUDA modules and a persistent stream.

CUDA stream wrapper for asynchronous kernel execution.

`<function=NAME>` / `<function NAME>` attribute-style calls.