Add `element` as its own singleton set. Returns `false` if it was already present.

Consume `cnt` bytes, updating line/column tracking.

Borrow as an object's entry slice, if it is one.

Snapshot `n_elements` BF16 values starting at `ptr + byte_offset`, widen to f32 little-endian, write to `<ATLAS_OP_DUMP>/atlas_op_L{layer_idx}_{op}.bi

Absolute path of the dump file (for startup logging).

Construct a `MetalGpuBackend` for tests, or return `None` if the host can't open a Metal device. CI runners (especially virtualised macOS hosts on Git

Outgoing edges of `state`.

The underlying FSM.

Finish building and return the [`GrammarData`], setting the root rule to the rule with `root_rule_name`. Returns `Err` if the named rule is not prese

Reserve the snapshot slot the next boundary should be written into, registering the `(token_position, slot)` entry. When the ring is full this **evic

Add a `RuleRef` expr pointing at `rule_id`.

Whether an object contains `key`.

Number of states.

Build a [`ParseError::Parse`] anchored at `peek(delta_element)`.

Set / clear token id `token`. Panics if `token >= vocab_size`.

Load a `(.weight, .scales, .biases)` triplet from a parsed safetensors blob and upload to the GPU. `base` is the tensor name minus the suffix (e.g. `"

The start state id.

Get or initialize the global registry. First call loads all PTX modules and creates the persistent stream. Subsequent calls return the cached registr

Build an optimized, FSM-accelerated grammar from an EBNF string.

Vocabulary size (determines bitmask width).

Find deepest snapshot matching session within matched_tokens range.

Free the three GPU buffers backing this weight. Idempotent if the pointers are null. Call this at model teardown — there's no Drop because `MlxInt8Wei

Remove every element.

True if `element` has been added.

Emit a token for an active sequence (stream + bookkeeping). Per OpenAI spec, stop/EOS tokens are NOT streamed to the client — the returned text must

Add an `EmptyStr` expr (the empty string `""`).

Extract as FP8, if this weight is that variant.

Finalize: close any trailing static run and return the segments.

True if `state` is accepting.

Extract as NVFP4, if this weight is that variant.

The root rule id (`-1` if unset).

Sequential [Q|K|V|Z] after deinterleaving.

W4A16 GEMV (M=1): C = A @ dequant(B) for single-row activations. A: [1, K] BF16, B: NVFP4 packed, C: [1, N] BF16. 4 outputs/block, 64 threads (2 warp

Layer type for a given layer index. Falls back to full_attention_interval if layer_types is empty.

Build a grammar from a structural-tag format object (the `format` body), wrapping it in the full `structural_tag` document.

Parse a grammar string with the default root rule name `"root"`.

A cache-enabled compiler bound to [`small_tokenizer`].

Build a [`RegexError`] at the current 1-based position — the C++ `RaiseError` offset is `current_ - start_ + 1`.

Batched expert up projection output.

Get a weight tensor by name. Fails fast if not found.

Record the elapsed time since `since` under `phase`. No-op when disarmed.

W4A16 GEMM: C = A @ dequant(B). A: [M, K] BF16 activations B: NVFP4 packed weights (E2M1 + FP8 scales + FP32 per-tensor scale) C: [M, N] BF16 output

Add a rule with the given `name` and `body_expr_id`.

Borrow as a bool, if it is one.

Number of rules.

Apply schema-driven type coercion to all calls in `calls`. Matches each call to its [`ToolDefinition`] by name, then rewrites JSON-string argument va

F73 (2026-04-29): test wrapper that defaults the new `inside_envelope: &mut bool` parameter. Tests in this module that pre-date F73 don't exercise env

Deterministic pseudo-random bf16-representable test value in ~[-2, 2].

Fill a fresh bitmask for `m` and return the set of accepted token ids. Panics if the matcher has terminated.

Create a new CUDA stream (for overlapping prefill with decode).

Batched expert gate projection output.

Load an FP8 E4M3 block-scaled checkpoint weight as a native [`Fp8Weight`]. The FP8 checkpoint stores: - `{prefix}.weight`: FP8E4M3 tensor [N, K] - `{

OpenAI-compatible error with optional `param` (field path like `messages[0].role`) and `code` (e.g. `"context_length_exceeded"`).

Feed a text delta. Returns events to emit (content or tool calls). Emits incremental ToolCallStart/ToolCallDelta/ToolCallEnd events so clients see too

Convert a regex string to an EBNF grammar string. When `with_rule_name` is `true` the result is wrapped as `root ::= <body>\n`; otherwise just the ru

Allocate a free block. Returns block index.

Advance one codepoint.

Enqueue `state` for predict/complete unless already visited. Returns true if it was newly enqueued.

Broadcast a u32 command from rank 0 to all ranks. Rank 0 writes `val` to GPU buffer and broadcasts. Other ranks receive the value and return it.

Batched expert down projection output.

Whether the payload is empty.

The current codepoint (`*current_`), or `0` at the terminator — matching the C-string null terminator the C++ relies on.

Advance every scanable state by input byte `ch`. Returns true if `ch` is accepted by at least one state. When `ch` is rejected the parser state is le

Convert NCCL result to anyhow::Result.

Compile a grammar from an EBNF string. Port of `GrammarCompiler::CompileGrammar(ebnf, root_rule_name)`.

Fill the allowed-token bitmask for the next decode step. Returns `true` if the bitmask constrains at least one token (i.e., is not all-ones). When `f

Flush remaining buffer (call at stream end). Also attempts bare `<function>` detection as a last resort.

Decode-path matvec: `y = self_dequant @ x`. `x` must be BF16 `[in_features]`; `y` must be a BF16 buffer with at least `out_features` slots. Runs the f

NVFP4 data section bytes per block (packed E2M1 nibbles).