Generate audio from a chatml sample. Args: chat_ml_sample: A chatml sample. max_new_tokens: The maximum numbe

Initialize codebook. Args: data (tensor): [B * T, D].

Memory-efficient K-means clustering. Args: samples (tensor): shape [N, D] num_clusters (int): number of centroids. nu

Merge input_ids with audio features into final embeddings. Args: audio_features_embed (`torch.Tensor` of shape `(num_audios, max_aud

Convert Chinese (full-width) punctuation marks to English (half-width) equivalents.

Prepare the ChatML DataFrame.

Chunk the text into smaller pieces. We will later feed the chunks one by one to the model. Parameters ---------- text : str The t

Prepare the context for generation. The context contains the system message, user message, assistant message, and audio prompt if any.

Collate the input data with support for long audio processing.

Retrieve a dataset sample by index.

Initialize the HiggsAudioServeEngine, a serving wrapper for the HiggsAudioModel. The model, tokenizer, and audio tokenizer will be do

:param original_embedding: An instance of nn.Embedding (the original embedding layer). :param freeze_until_idx: The index up to which

Retrieve a sample by iterating through the dataset.

r""" Generates sequences of token ids for joint text/audio models using **multinomial sampling**. This function may also be revised t

Wrap an async generator with the first element.

Average a dictionary of metrics across all workers, using the optional `count` as unormalized weight.

Decode the given codes to the quantized representation.

Perform all-to-all before and after the attention function.

Slice the corresponding cos and sin chunks for rope.

Return the current range width.