Build VarBuilder for CausalSelfAttention. Called as vb.pp("self_attn") by the load fn.

Read a safetensors header and return per-tensor byte sizes. The safetensors format stores an 8-byte LE header length followed by a JSON object mapping

Save PCM f32 samples as a WAV file (16-bit PCM, mono).

Get the inference timesteps.

Build a minimal safetensors file with the given header JSON.

Return a set of all layer names assigned to workers in this topology.

Helper: connect to addr, do Hello handshake, return stream + WorkerInfo.

Create a VarBuilder that loads tensors in their native stored dtype. F8E4M3 tensors are kept in F8E4M3 on the target device. # Safety Inherits the m

Build VarBuilder for a full Transformer block (pre-norm + attention + MLP).

Total estimated TFLOPS across all GPUs.

Return the cached cosine value for the given position and sequence length. When `device` differs from the cache's own device, the result is copied to

Predict EOS probability. Input: (batch, hidden_size) LLM hidden state. Output: (batch, 1) probability (after sigmoid).

Forward pass with external cache (for VibeVoice TTS).

Create a safetensors file with `n` experts, each with gate/up/down projections.

Automatically assign layers to nodes using the provided [`Strategy`]. This is used when a topology file specifies worker addresses but leaves `layers

Bind a TCP listener on localhost with an OS-assigned port.

Create a temporary safetensors file with given tensors. Returns the temp directory (keeps the file alive).

Decode WAV bytes and return mono PCM f32 samples at the given target sample rate. Same processing as [`load_wav_mono`] but operates on in-memory byte

Detect available compute devices on this system. Only reports NVIDIA GPUs when the `cuda` feature is compiled in, and Metal on macOS when the `metal`

Estimate average transformer layer size in bytes from safetensors files. For sharded models, reads each shard's header to compute exact per-tensor by

Load the safetensors files for a model from the hub based on a json index file.

Build VarBuilder for MLP.

Get the attention mask for the given sequence length.

Run one mock-worker session: accept a connection, optionally authenticate, handle Hello, then echo tensors for SingleOp/Batch until the client disconn

Run a complete diffusion step, managing solver order automatically. `step_idx`: 0-based index into timesteps `x0_buffer`: mutable buffer of previous

GatedDeltaNet config for Qwen3.5 linear attention benchmarks.

Check a single directory and return its model status, or None if it's not a model dir.

Compute HMAC-SHA256(key, data).

Detect quantization strategy from a model's config.json.

Downloads all required model files from HuggingFace Hub. Returns the local directory path containing the downloaded files. Checks local cache first —

Group normalization: `(x - mean) / sqrt(var + eps) * weight + bias` per group. Matches candle_nn::GroupNorm::forward() — F32 promotion for F16/BF16. I

Create a VarBuilder with the tensors loaded from the index.

Adaptive LayerNorm: norm(x) * (1 + scale) + shift LayerNorm computes in F32, then casts back. Scale/shift arithmetic in input dtype (BF16).

Like `process_kv` but caps the KV cache to `window` tokens (sliding window attention).

SwooshR(x) = log(1 + exp(x - 1)) - 0.08*x - 0.313261687 Matches Python's SwooshRForward with offset=1.

Top-K selection: returns `(values, indices)` for the largest K elements along the last dimension. Uses partial sort O(N + K log K) for large N.

Helper: write tensors to a temp .safetensors file, return path.

Write u32 data into the buffer.

Master side of the mutual authentication. Called on the TCP stream immediately after connection, before any Cake protocol messages.

Cache a dispatch output buffer by the returned Tensor's TensorId. If the next dispatch consumes this tensor, `get_or_upload` will find the buffer here

Clear the cache.

Detect the system hostname.

Check the HuggingFace cache for an already-downloaded complete model. Returns the snapshot directory path if found with all shards present.

Forward through LM layers with cache. Sets the cache in fwd_ctx before each call. Local layers use it for KV caching; remote layers (Client) ignore it

Create a VarBuilder that only loads safetensors shards containing tensors for the given layer prefixes. Workers use this to skip shards that only cont