Feed multiple embedding vectors (prefill). Automatically chunks into sub-batches of ``n_batch`` tokens.

Feed a single embedding vector at the given position.

Reconstruct audio from a given .dac file Parameters ---------- obj : Union[str, Path, DACFile] .dac file location

Simulation LLM stream output text delta. The streaming response of OpenAI/vLLM is replaced in the real scenario.

Simulate streaming text deltas from an LLM. Each iteration yields `chunk_chars` characters with a delay of `delay_s` seconds. In real-wor

tensor: torch.Tensor of shape [batch_size, seq_len] C: scalar value to match Returns: torch.Tensor of shape [batch_size] with last indice

Mark text stream end and emit all remaining audio chunks (including flush).

与 gradio AudioFrameDecoder.flush 完全一致。

Given the quantized codes, reconstruct the continuous representation Parameters ---------- codes : Tensor[B x N x T]

Generate speech from text. Returns: waveform: float32 array at 24 kHz When ``low_memory`` is enabled, real-time streamin

Get the hidden state for the i-th output token. Returns a copy as float32 array of shape (n_embd,).

Get the text logits for the i-th output token. Returns a copy as float32 array of shape (n_vocab,).

Copied from https://github.com/hehao13/CameraCtrl/blob/main/inference.py

Device-level GPU memory used (all processes), in MB. This is the reliable metric — works on all driver/container combos.

Return (gpu_name, total_mb).

Initialize the delay state from the prefill input_ids. Args: input_ids: (S, 33) — the full prompt sequence (already packed).

Load the MOSS-Audio-Tokenizer codec model onto the given device.