Converts a space-separated string of tokens to a list of ids.

source_encoding: [T, C] lengths, list of int, [T,], how many times each token should repeat return: expanded_encoding: [T_expand,

FP16-compatible function that fills a tensor with -inf.

FP16-compatible function that fills a tensor with -inf.

x : torch.Tensor, shape = (batch_size, n_mels, n_ctx) the mel spectrogram of the audio

x : torch.LongTensor, shape = (batch_size, <= n_ctx) the text tokens xa : torch.Tensor, shape = (batch_size, n_audio_ctx,

Perform a forward pass through the TransformerBlock. Args: x (torch.Tensor): Input tensor. start_pos (int):

Apply layer normalization. :param torch.Tensor x: input tensor :return: layer normalized tensor :rtype torch.Tensor

Input is expected to be of size [bsz x seqlen].

Apply rotary embeddings to an array. :param input: Array to apply the rotary embeddings to. :param positions: positions of t

Input shape: Time x Batch x Channel Args: key_padding_mask (ByteTensor, optional): mask to exclude keys that are

Input shape: Time x Batch x Channel Args: key_padding_mask (ByteTensor, optional): mask to exclude keys that are

Example (no batch dim version): 1. dur = [2,2,3] 2. token_idx = [[1],[2],[3]], dur_cumsum = [2,4,7], dur_cumsum_prev

Input is expected to be of size [bsz x seqlen].

Input shape: Time x Batch x Channel Args: key_padding_mask (ByteTensor, optional): mask to exclude keys that are

:param txt_tokens: [B, T] :return: { 'encoder_out': [B x T x C] }

:param h: [B, T, H] :param seg_ids: [B, T] :return: h_ph: [B, T_ph, H]

The `MultiHeadAttention` module optionally accepts `kv_cache` which stores the key and value tensors calculated for the previous posi

Make mask tensor containing indices of non-padded part. Args: lengths (LongTensor or List): Batch of lengths (B,). xs (Tensor, opt

Maximum number of supported positions.

Maximum number of supported positions.

:param attn_logits: [n_layers, B, n_head, T_sp, T_txt] :return:

Write vocab file to disk. Vocab files have one token per line. The file ends in a newline. Reserved tokens are written to the vocab f