* Mirrors the JSON status produced by the Rust worker library. * * Stages: "idle", "starting", "discovery", "connected", "authenticated", *

Arguments for audio/TTS generation.

Output from audio generation.

Power-of-2 buffer pool for output and activation buffers.

BypassModule: combines original (pre-downsample) with processed (upsampled) output. `output = bypass_scale * src_orig + (1 - bypass_scale) * processed

Chat history format: LLaMA-3 native or ChatML fallback (e.g. for SmolLM2).

ChatML chat history shared by Qwen2, Qwen3.5, and other ChatML-based models.

Abstraction over compute backends (CPU, CUDA, Metal, Vulkan). Each method has a CPU-based default via [`CpuBackend`]. GPU backends override specific

Default stateless sharding strategy that assigns layers proportionally to each worker's TFLOPS, capped by per-GPU VRAM.

Handle for a running discovery listener. Must be kept alive for the worker to respond to discovery queries. Dropping this handle signals the listener

EXAONE 4.0 model (ExaoneForCausalLM).

Simple LRU cache for dequantized expert weights. Avoids re-dequantizing the same popular experts across tokens.

Pre-computed tensor names for a single expert (avoids format! on hot path).

Trait for providing expert weight tensors on demand. MoE `forward()` methods call `get_expert(idx)` to obtain the three weight matrices (gate, up, do

Falcon3 model (FalconForCausalLM). Standard GQA + SwiGLU transformer, same block structure as LLaMA.