Create adapter for the base model. Args: model (torch.nn.Module): Base model to be adapted. adapter_type (str): Name of adapter

Make batch of inputs with left padding for LM scoring. Args: hyps_seq: Hypothesis sequences. Returns: : Padd

Create query with both text and audio inputs for Qwen2-Audio. This is the core tokenization process from the original example.

Crop the input tensor. Args: x (torch.Tensor): Input tensor of shape [B, C, T]. yin_start (int): Starting point of th

End detection. described in Eq. (50) of S. Watanabe et al "Hybrid CTC/Attention Architecture for End-to-End Speech Recognition" :param e

Reset the positional encodings.

Forward one step. Args: tgt: input token ids, int64 (batch, maxlen_out) tgt_mask: input token mask, (batch, maxlen_o

Transform query, key and value. Args: query (torch.Tensor): Query tensor (#batch, time1, size). key (torch.Tensor): K

r"""Generate t-SNE plot for language embeddings with labels directly on the points. Args: lang_to_embds_dic (dict): Dictionary mapping la

Generate a list of long utterances from a list of short utterances.

Return activation function. Args: activation_type: Activation function type. ftswish_threshold: Threshold value for FTSwish activ

Return the convolution module parameters. Args: input_size: Module input size. last_conv_size: Last convolution size for module o

Return the configuration as dict. This method is used by print_config()

Returns keys2 e.g. loss,acc.

Get number of frames. Examples: >>> get_num_frames_writer('ark,t:num_frames.txt')

Get the number of nodes. Use for "multiprocessing distributed" mode. RANK equals to the Node id in this case and the real Rank is set as

Return the global Dask cluster configuration.

Return the output size of the VGG frontend. :param in_channel: input channel size :param out_channel: output channel size :return: output

Halve a minibatch-size string, as would be validated by validate_minibatch_size_str (see docs for that). This halves all the integer elements

Compute and return Word Error Rate (WER) and Character Error Rate (CER) metrics for multiple ASR systems (ESPnet, OWSM, Whisper) using the Ve

Build partial or final results from the hypotheses. Args: nbest_hyps: N-best hypothesis. Returns: results: R

Run inference. Args: text (LongTensor): Batch of padded character ids (B, Tmax). text_lengths (LongTensor): Batch of

Transforms tensor/list/tuple into tuple.

check character is Mandarin

Load a packaged default config without resolving interpolations. This helper reads a default config bundled in an ESPnet recipe package. The

Load wavfile given recid, start time and end time.

Obtain Logmel from signal.

Compute the perplexity of the given text using a specified model from the `evaluate` library (default: GPT-2). Args: LLM_Output

Convert single raw data dict into training-ready format. Applies chat template, tokenizes content, adds special tokens, and creates l

Wraps modules and applies quantization noise to the weights for subsequent quantization with Iterative Product Quantization as described in "

Recombine hypotheses with same label ID sequence. Args: hyps: Hypotheses. Returns: final: Recombined hypothe

Reconstructs the list of tensors from the flattened output.

Resolve a requested stage list against the available stages. The special token ``"all"`` expands to all known stages in order. Otherwise, the

Save the ESPnet config used for training to the output directory.

Update properties. Args: **kwargs: Key-value dict that contains all properties with their new values. Unknown pro

Set CTC segmentation parameters. Parameters for timing: time_stamps: Select method how CTC index duration is estimated, and

Return a new dataset shard corresponding to the given index. This method must be implemented by subclasses to return a subset of the data

Speech2Text streaming call. Args: speech: Chunk of speech data. (S) is_final: Whether speech corresponds to the final

Train the model, training the tokenizer first if needed. This stage checks for a cached tokenizer model and runs tokenizer training b

Scan all files in the archive file and return as a dict of files. Examples: tarfile: model.pth some1.file

Calculate and return Self BLEU-2, Auto BLEU-2 and VERT-2 metrics for a list of LLM responses. Args: LLM_response_arr (List[str])