Helper class to manage the options for variable objects and its configuration. Example: >>> class A: ... def __init__(self, foo=3):

Layer normalization module. Args: nout (int): Output dim size. dim (int): Dimension to be normalized.

CTC module. Args: odim: dimension of outputs encoder_output_size: number of encoder projection units dropout_rate: dropou

ESPnet3 LightningModule wrapper for model training and data integration. This wrapper keeps the common ESPnet3 model contract unchanged: ```

Simple implementation of ArgumentParser supporting config file This class is originated from https://github.com/bw2/ConfigArgParse, but this

Functor class of common_collate_fn()

Reporter class. Examples: >>> reporter = Reporter() >>> with reporter.observe('train') as sub_reporter: ... for batc

Writer class to create kaldi like data directory. Examples: >>> with DatadirWriter("output") as writer: ... # output/sub.txt

Conformer encoder module. Args: input_size (int): Input dimension. output_size (int): Dimension of attention. attention_h

Conv1d module with customized initialization.

Encoder module of Spectrogram prediction network. This is a module of encoder of Spectrogram prediction network in Singing Tacotron, which de

Hypothesis data type.

STFT encoder for speech enhancement and separation

Beam search implementation.

Default hypothesis definition for Transducer search algorithms.

Multi-Head Attention layer inheriting SequenceModule. Comparing default MHA module in ESPnet, this module returns additional dummy state and

Organizes training, validation, and test datasets into a unified interface. This class constructs combined datasets for training and validation,

Combines multiple datasets into a single unified dataset-like interface. This class supports seamless access to multiple datasets as if they were

Length bonus in beam search.

Transformer encoder.

Branchformer encoder module.

Discriminator adversarial loss module.

A wrapper around Lightning's Trainer to provide ESPnet3-specific integration. This trainer ensures compatibility with ESPnet's dataloader, callba

Generator adversarial loss module.

Transducer joint network module. Args: output_size: Output size. encoder_size: Encoder output size. decoder_size: Decoder

Writer class for 'score.scp' Examples: key1 /some/path/score.json key2 /some/path/score.json key3 /some/path/score.json

Pytorch Dataset class for ESPNet. Examples: >>> dataset = ESPnetDataset([('wav.scp', 'input', 'sound'), ...

Speech enhancement or separation Frontend model

Writer class for a scp file of numpy file. Examples: key1 /some/path/a.npy key2 /some/path/b.npy key3 /some/path/c.npy

Feature matching loss module.

STFT decoder for speech enhancement and separation

Convolutional 2D subsampling (to 1/4 length). Args: idim (int): Input dimension. odim (int): Output dimension. dropout_ra

Multiconvformer encoder module. Link to the paper: https://arxiv.org/abs/2407.03718 Args: input_size (int): Input dimension.

Basic Conv2D + activation + instance norm building block.

Factory for creating data iterators for SpeechLM training. This class manages batching, data sharding across GPUs, and provides DataLoader in

Builder class for constructing training and validation DataLoaders in ESPnet3. This class provides a unified interface for setting up PyTorch or

Phonemizer module for various languages. This is wrapper module of https://github.com/bootphon/phonemizer. You can define various g2p modules

Positional encoding. Args: d_model (int): Embedding dimension. dropout_rate (float): Dropout rate. max_len (int): Maximum

Positionwise feed forward layer. Args: idim (int): Input dimenstion. hidden_units (int): The number of hidden units. drop

E-Branchformer encoder module.

Pytorch Dataset class for ESPNet. Examples: >>> dataset = IterableESPnetDataset([('wav.scp', 'input', 'sound'), ...

Reader class for 'wav.scp'. Examples: wav.scp is a text file that looks like the following: key1 /some/path/a.wav key2 /

Speech2Text class Examples: >>> import soundfile >>> speech2text = Speech2Text("asr_config.yml", "asr.pth") >>> audio, ra

VITS module (generator + discriminator). This is a module of VITS described in `Conditional Variational Autoencoder with Adversarial Learning

Convolutional 2D subsampling (to 1/6 length). Args: idim (int): Input dimension. odim (int): Output dimension. dropout_ra

Convolutional 2D subsampling (to 1/8 length). Args: idim (int): Input dimension. odim (int): Output dimension. dropout_ra

Discrete audio I/O using combined codec and SSL tokenizers. This class handles audio encoding/decoding using both: 1. Codec tokens (acoustic/

E-Branchformer encoder module. Compared to the original encoder in e_branchformer_encoder.py, this variant supports additional cross-attentio

Default hypothesis definition for Transducer search algorithms.

PQMF module. This module is based on `Near-perfect-reconstruction pseudo-QMF banks`_. .. _`Near-perfect-reconstruction pseudo-QMF banks`:

Postnet module for Spectrogram prediction network. This is a module of Postnet in Spectrogram prediction network, which described in `Natural

RNN module. :param int idim: dimension of inputs :param int elayers: number of encoder layers :param int cdim: number of rnn units (resul

Writer class for 'wav.scp' Args: outdir: scpfile: format: The output audio format multi_columns: Save multi chann

Dict-like text reader supporting plain and JSONL formats. Plain format: <id> <text content> JSONL format: {"id": "<id>", "text": "<text conte

Raised when the utt is too short for subsampling. Args: message (str): Message for error catch actual_size (int): the short size

Transformer encoder module. Args: input_size: input dim output_size: dimension of attention attention_heads: the number o

Decoder interface wrapper for CTCPrefixScore.

Convolutional 2D subsampling (to 1/2 length). Args: idim (int): Input dimension. odim (int): Output dimension. dropout_ra

Encoder layer module. Args: size (int): Input dimension. self_attn (torch.nn.Module): Self-attention module instance.

attention-based label smoothing loss for S2ST.

Beam search implementation for Transducer.

Hugging Face Transformers PostEncoder.

1x1 Conv1d with customized initialization.

Similar to Conv2dSubsampling module, but without any subsampling performed. Args: idim (int): Input dimension. odim (int): Output

Conventional frontend structure for ASR. Stft -> WPE -> MVDR-Beamformer -> Power-spec -> Log-Mel-Fbank

Apply global mean and variance normalization TODO(kamo): Make this class portable somehow Args: stats_file: npy file norm_me

Default hypothesis definition for Transducer search algorithms. Args: score: Total log-probability. yseq: Label sequence as integ

Label-smoothing loss. :param int size: the number of class :param int padding_idx: ignored class id :param float smoothing: smoothing rat

Length Regulator

Fullscorer for ngram.

Partialscorer for ngram.

Conv1d with some builtin handling of asymmetric or causal padding and normalization.

SI-SNR (or named SI-SDR) loss A more stable SI-SNR loss with clamp from `fast_bss_eval`. Attributes: clamp_db: float cla

SeparateSpeech class Examples: >>> import soundfile >>> separate_speech = SeparateSpeech("enh_config.yml", "enh.pth") >>>

Build iterator for each epoch. This class simply creates pytorch DataLoader except for the following points: - The random seed is decided acc

Sequential RNNLM. See also: https://github.com/pytorch/examples/blob/4581968193699de14b56527296262dd76ab43557/word_language_model/model.p

Text cleaner. Examples: >>> cleaner = TextCleaner("tacotron") >>> cleaner("(Hello-World); & jr. & dr.") 'HELLO WORLD,

(RNN-)Transducer decoder module. Args: vocab_size: Output dimension. layers_type: (RNN-)Decoder layers type. num_layers:

A custom callback for weight averaging over the top-K checkpoints. This can be useful to smooth out fluctuations in weights across the best-perfo

Convolutional encoder for speech enhancement and separation