Send prompt + n_gen request, return generated token ids.

Generate a prompt that's approximately n_tokens long.

Get the current routing stats (if tracked). Returns nullptr if the backend does not support routing stats or they are not enabled.

Read a u32 array key into a vector (empty if not found or not an array).

L1 distance between an empirical histogram of GPU draws and the analytic distribution. Pulls the uniform from the same RNG family the CPU path uses.

Per-call latency microbench (gated by env DFLASH_SAMPLER_BENCH=1). Isolates the three regimes that explain the end-to-end numbers: the CPU chain, the

Run completion + task['test'] + check_call in a subprocess with a SIGALRM-based timeout. Returns dict with ok/err. If completion starts with

Returns true if `s`, after skipping leading whitespace, starts with kBillingHeader.

Residency policy from DFLASH_KVFLASH_POLICY (--kvflash-policy): "lru" forces recency-only paging (no drafter probe, no scorer); anything else (default

"qk": target-QK residency scoring (kvflash_qk.h) — pooled post-RoPE keys vs the current decode query, no drafter involved.

Chunked prefill loop on top of the shared laguna_step() helper. Reports total prefill time and the argmax / logit at the LAST chunk.

---- Cache snapshot helpers (prefix-cache slots) ------------------------ laguna_snapshot_alloc: build a parallel set of K/V tensors RIGHT-SIZED to s

---- Single-graph hybrid decode forward -----------------------------------

Map LLAMA_FTYPE_* int → operator-friendly tag (Q4_K_M, IQ4_XS, BF16, …). Kept inline so we don't pull in llama.h here — those enum values are part of

Resolve the draft's architecture scalars. config.json (next to the safetensors) is authoritative for the transformer hparams; the tensor shape

Load multiple routing files and return token-shuffled concatenation. Each file is sliced into per-token blocks of ``num_layers`` records. Tok

Load a single binary routing file into numpy arrays.

Real-content filler: raw little-endian i32 token stream. Ids are folded below 100000 (drafter-scoreable, same fold as KvFlashDrafterScorer / make_prom

Load Qwen3.5-0.8B weights and optional GB10 NVFP4 decode weights.

Load Qwen3.5-0.8B weights (bf16 on SM>=80, fp16 on SM<80).

Top-1 ListNet: CE between softmax(scores) and the normalized relevance.

Pairwise RankNet logistic loss over (selected, unselected) score pairs. `sel` is the padded (N, 8) selected-expert index matrix (-1 = empty slot)