log_softmax of frame activations Args: Tensor hs_pad: 3d tensor (B, Tmax, eprojs) Returns: torch.Tensor: log

Stable log add

Add <sos> and <eos> labels. Args: ys_pad (torch.Tensor): batch of padded target sequences (B, Lmax) sos (int): index of <sos>

Make mask tensor containing indices of padded part. See description of make_non_pad_mask. Args: lengths (torch.Tensor): Batch of len

Reverse padding for the list of tensors. Args: ys_pad (tensor): The padded tensor (B, Tokenmax). ys_lens (tensor): The lens of to

Construct a Squeeze and Excite Module. :param in_channels: Number of input channels. :param rd_ratio: Input channel reduction ratio.

[H, W, 3] uint8 → [3, H, W] float32 tensor，并归一化到 [0, 1]。

Create mask for subsequent steps (size, size). This mask is used only in decoder which works in an auto-regressive mode. This means the curre

Forward just one chunk Args: xs (torch.Tensor): chunk input offset (int): current offset in encoder output time stam

Forward one step. This is only used for decoding. Args: memory: encoded memory, float32 (batch, maxlen_in, feat)

Apply beam search on attention decoder Args: speech (torch.Tensor): (batch, max_len, feat_dim) speech_length (torch.

Args: infos (dict or None): any info you want to save.

Method to fuse batchnorm layer with preceeding conv layer. Reference: https://github.com/DingXiaoH/RepVGG/blob/main/repvgg.py#L95 :p

argmax of frame activations Args: torch.Tensor hs_pad: 3d tensor (B, Tmax, eprojs) Returns: torch.Tensor: arg

Apply attention rescoring decoding, CTC prefix beam search is applied first to get nbest, then we resoring the nbest on atten

从 FACE_CROP_SIZE 大小的 crop 中取中心的 FACE_INNER_SIZE 部分。

If a sequence is finished, all of its branch should be <eos> Args: pred (torch.Tensor): A int array with shape (batch_si

If a sequence is finished, we only allow one alive branch. This function aims to give one branch a zero score and the rest -inf score. A

Train one epoch

Args: input_size (int): input dim output_size (int): dimension of attention attention_heads (int): the nu

Args: input_size (int): input dim output_size (int): dimension of attention attention_heads (int): the nu

Construct an PositionalEncoding object.

Helper method to construct conv-batchnorm layers. :param kernel_size: Size of the convolution kernel. :param padding: Zero-padding s

CTC prefix beam search inner implementation Args: speech (torch.Tensor): (batch, max_len, feat_dim) speech_length (t

CTC prefix beam search inner implementation Args: speech (torch.Tensor): (batch, max_len, feat_dim) speech_length (t

静音状态/音频不够时返回的"占位帧"。每 5 次调用输出 1 帧图。

Apply optional mask for encoder. Args: xs (torch.Tensor): padded input, (B, L, D), L for max length mask (torch.Tensor): mask fo

从关键点算出包含嘴部的正方形 bbox (xmin, ymin, xmax, ymax)。 选点规则沿用原作者：横向以 #1 / #31 关键点为左右边界，纵向以 #52 为上边界， 并强制为正方形（边长等于横向宽度）。

裁切 bbox 区域并 resize 到 FACE_CROP_SIZE。

Apply CTC greedy search Args: speech (torch.Tensor): (batch, max_len, feat_dim) speech_length (torch.Tensor): (batch

Apply CTC prefix beam search Args: speech (torch.Tensor): (batch, max_len, feat_dim) speech_length (torch.Tensor): (

Cross validation on

Apply forward pass.

Compute attention context vector. Args: value (torch.Tensor): Transformed value, size (#batch, n_head, time2, d_k

Forward input chunk by chunk with chunk_size like a streaming fashion Here we should pay special attention to computation cache

Transform query, key and value. Args: query (torch.Tensor): Query tensor (#batch, time1, size). key (torch.Tensor): K

取以 index 为中心、半径 half_window 的音频特征窗口，越界处用 0 填充。 返回 shape [2*half_window, ...] 的 tensor，dtype 跟 features 保持一致。

Return activation function.

便捷函数：读图 + 读关键点 + 算 bbox + 裁切，返回 FACE_CROP_SIZE 大小的 crop。

把 FACE_INNER_SIZE 大小的人脸图嘴部矩形区域涂黑（原地修改并返回）。

Perform padding for the list of tensors. Args: xs (List): List of Tensors [(T_1, `*`), (T_2, `*`), ..., (T_B, `*`)]. pad_value (f

从 .lms 文件读关键点。每行 'x y'，返回 int32 ndarray [N, 2]。

根据 ASR 模式 reshape 音频特征到 UNet 期望的形状。

Create mask for subsequent steps (size, size) with chunk size, this is for streaming encoder Args: size (int): size of mask

Calculate accuracy. Args: pad_outputs (Tensor): Prediction tensors (B * Lmax, D). pad_targets (LongTensor): Target label tensors

Construct an MultiHeadedAttention object.

Construct an EncoderLayer object.

随机取另一帧作为参考。

加载当前帧，返回 (target_T, masked_T)。

从 168x168 crop 构造 unet 的 6 通道输入和回贴用 crop_ori。

#assert speech.shape[0] == speech_lengths.shape[0] #assert decoding_chunk_size != 0 #batch_size = speech.shape[0] #en

Method to obtain re-parameterized kernel and bias. Reference: https://github.com/DingXiaoH/RepVGG/blob/main/repvgg.py#L83 :return: T

Load the json format cmvn stats file and calculate cmvn Args: json_cmvn_file: cmvn stats file in json format Returns: a num

Load the kaldi format cmvn stats file and calculate cmvn Args: kaldi_cmvn_file: kaldi text style global cmvn file, which is

把网络预测的 inner 区域贴回原 face crop，再 resize 回原 bbox 大小，覆写 img。

返回 (img_concat_T, crop_img_ori, bbox, original_size)。

读 .lms → 返回 (xmin, ymin, xmax, ymax)，与训练裁切口径一致。

从 audio_queue 取一段做 fbank + wenet encoder，返回新增的音频特征。

对当前帧做一次 unet 推理并把结果贴回原图。