Encode given audio data and return quantized latent codes Parameters ---------- audio_data : Tensor[B x 1 x T] Au

Decode given latent codes and return audio data Parameters ---------- z : Tensor[B x D x T] Quantized continuous

Broadcast the tensors from the given parameters to all workers. This can be used to ensure that all workers have the same model to start with.

Divide a tensor among the sequence parallel ranks

Gather tensors and concatenate them along a dimension

Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`,

Implement the delay pattern proposed in "Simple and Controllable Music Generation", https://arxiv.org/pdf/2306.05284 In the delay pattern, each c

Preprocess the ChatML sample to get the tokens for the text part. Args: sample (ChatMLSample): The ChatML sample to preprocess. t

Push the given symbol on the stream, flushing out bits if possible. Args: symbol (int): symbol to encode with the AC.

Receives tokens and processes them as either text or audio tokens. For text tokens, decodes and caches them until complete words are

Embed the audio ids Args: audio_ids: torch.LongTensor of shape (num_codebooks, audio_in_total_length) Returns:

Gather tensors and concatenate them along a dimension.

Prepare the KV cache for the current sequence length.

Turn the given PDF into a quantized CDF that splits [0, 2 ** self.total_range_bits - 1] into chunks of size roughly proportional to the PDF.

Capture CUDA graphs for the model's forward pass with different KV cache lengths. Args: past_key_values: List of KV caches to cap

Return bandwidth per quantizer for a given input sample rate.

Return n_q based on specified target bandwidth.

Pull a symbol, reading as many bits from the stream as required. This returns `None` when the stream has been exhausted. Args:

Convert samples encoded with delay pattern back to the original form. Args: data (:obj:`torch.Tensor`): The data with delay p

:param original_linear: The original nn.Linear layer. :param freeze_until_idx: The index up to which the rows of the weight matrix ar

Apply the audio tower to the audio features

Computes the output length of the convolutional layers and the output length of the audio encoder

Process long audio and duplicate corresponding audio tokens. Args: input_ids: Input token ids audio_idx: Index of the

Sample audio tokens and its corresponding text tokens from the logits

Sample text tokens from the logits

Scatter tensors.

Update the model kwargs for each step.

The whisper encoder does not support zero-shape tensor by default due to the following implementations key_states = self._shape(self.k_proj(c

Processes an audio signal from a file or AudioSignal object into discrete codes. This function processes the signal in short windows,

Reconstruct audio from a given .dac file Parameters ---------- obj : Union[str, Path, DACFile] .dac file location

Encode a content from a local file to base64 format.

Encode a content from a local file to base64 format.

Flushes any remaining text tokens and signals the end of generation.

Flush the remaining information to the stream.

Given the quantized codes, reconstruct the continuous representation Parameters ---------- codes : Tensor[B x N x T]