Check if all the given tokens are in token symbol table. Args: token_sym_table: Token symbol table that contains all the valid toke

NOTE(xcsong): The "units.txt" file is solely for adapting to the training API of Wenet and for quickly checking the corresponding tex

Export interface for c++ call, apply linear transform and log softmax before ctc Args: xs (torch.Tensor): encoder out

Cross validation on

...

download from url to dest

Dynamic batch the data until the total frames in batch reach `max_frames_in_batch` Args: data: Iterable[{key, feat, labe

Dynamic batch the data until the total frames in batch reach `max_frames_in_batch` Args: data: Iterable[{key, feat, labe

Dynamic batch the data until the total frames in batch reach `max_frames_in_batch` Args: data: Iterable[{key, feat, labe

Dynamic batch the data until the total frames in batch reach `max_frames_in_batch` Args: data: Iterable[{key, feat, labe

r""" Estimate the gradient of the model on the given dataset

When reaching the end of the decoded sequence, we need to finalize the matching, the purpose is to subtract the added bonus score for the

Identity with 4-D dataflow, input == output. Args: x (torch.Tensor): (batch, in_channel, 1, time) Returns: (t

Linear with 4-D dataflow. Args: x (torch.Tensor): (batch, in_channel, 1, time) Returns: (torch.Tensor): (batch

Subsample x with 4-D dataflow. Args: x (torch.Tensor): Input tensor (#batch, 1, mel_dim, time). Returns: torc

Compute scaled dot product attention. Args: q (torch.Tensor): Query tensor (#batch, size, 1, chunk_size). k (torch.Te

Compute convolution module. Args: x (torch.Tensor): Input tensor (#batch, channels, 1, chunk_size). cache (torch.Tenso

Forward function. Args: xs: input tensor (B, D, 1, L) Returns: output tensor, (B, D, 1, L)

Compute encoded features. Args: x (torch.Tensor): (#batch, size, 1, chunk_size) att_mask (torch.Tensor): Mask tensor

Forward just one chunk Args: xs (torch.Tensor): chunk input, with shape (b=1, 1, time, mel-dim), where `time ==

frame activations, without softmax. Args: Tensor x: 4d tensor (B, hidden_dim, 1, chunk_size) Returns: torch.T

Compute attention context vector. Args: value (torch.Tensor): Transformed value, size (#batch, n_head, time2, d_k

Export interface for c++ call, forward decoder with multiple hypothesis from ctc prefix beam search and one encoder output Args:

Forward just one chunk Args: xs (torch.Tensor): chunk input, with shape (b=1, time, mel-dim), where `time == (ch

Forward just one chunk Args: xs (torch.Tensor): chunk input offset (int): current offset in encoder output time stam

Chunk-based context mode Frontend + Encoder + Decoder + Calc loss Args: speech: (Batch, Length, ...) speech_l

Forward input chunk by chunk with chunk_size like a streaming fashion Here we should pay special attention to computation cache

Export interface for c++ call, give input chunk xs, and return output from time 0 to current chunk. Args: xs (torch.

Full context mode Frontend + Encoder + Decoder + Calc loss Args: speech: (Batch, Length, ...) speech_lengths:

Args: peaks: ctc peaks time stamp max_duration: max_duration of the sentence frame_rate: frame rate of every time stamp,

Generate a lexicon from a word list and token_sym_table. Args: token_sym_table: Token symbol table that mapping token to token ids.

Generate tokens from the given text file. Args: text_file: A file that contains text lines to generate tokens. Returns: R

Generate words from the given text file. Args: text_file: A file that contains text lines to generate words. Returns: Ret