Send data to an SSE client fd with a short (1s) timeout to avoid stalling the inference worker. Returns false if the send fails or times out.

Map safetensors dtype string to ggml type

Decode one token. Returns next token id.

Decode multiple NVFP4 steps without per-token host/device synchronization.

Strip leading </think> tags (with optional whitespace) from the start.

Sanity-check sparse attention at alpha < 1.0: The output should not be all zeros (basic liveness check). With alpha < 1.0 outputs will differ from den

hipcc path: use the type's own constructors / conversions

Synthesise a Codex/Claude-Code style user turn. Structure mimics what an agent client actually sends after a few tool calls have run (read_fi

Build a Codex-style user message padded to ~target_chars.

Extend [r0,r1] outward until both ends sit on a whitespace boundary. A token's text "starts on whitespace" if its decoded form begins with a

Extract a numeric answer from model output for GSM-style problems.

Forward request verbatim (no injection needed).

Group consecutive integers into [start,end] inclusive runs.

Load prompts for a bench suite from JSONL file.

Pack layer weights into device blob matching LayerWeights struct.

Pack layer weights into device blob matching LayerWeights struct.

Pack layer weights into device blob matching LayerWeightsNVFP4 struct.

Parse test_generate stdout → {decode_tps, n_gen}.

Parse test_dflash stdout into {prefill_s, decode_tps, al, stages{}}.

Read server stderr log if available.

GPU memory allocated since daemon spawn, in MiB. Priority order: 1. hipMemGetInfo delta — works on AMD unified-memory APUs (Strix Hal

Greedy subsequence match: returns positions in full_ids that align with kept_ids in order.

Run a single bench case via streaming to capture TTFT and detailed metrics. Always uses streaming to get: walltime, TTFT, prompt_tokens, completi

Run all prompts for a given bench suite.

Returns the argmax token id at the last prompt position via the daemon.

Score a GSM8K response. Returns (correct, detail_str).

Score a HumanEval response by executing the generated code against test cases. Extracts code from model output, appends the test harness, and run

Score a Math500 response. Returns (correct, detail_str).

Score a math response text against the gold answer. Extracts \\boxed{} answers (after </think> for thinking models), with fallbacks for **bol

Extract the model version token (e.g. '3.6') from a target filename.

Filesystem-safe slug for tokenizer cache keying.

Block until the daemon emits its explicit ready banner. The memory telemetry helpers are kept for diagnostics, but readiness is drive