Multi-headed attention from "Attention Is All You Need

Feed Forward Module follow pre-norm residual units and apply layer normalization within the residual unit and on the input before the first l

Feed Forward Module follow pre-norm residual units and apply layer normalization within the residual unit and on the input before the first l

When kernel size == 1 conv1d, this operation is termed in literature as pointwise convolution. This operation often used to match dimensions.

When kernel size == 1 conv1d, this operation is termed in literature as pointwise convolution. This operation often used to match dimensions.

This module produces sinusoidal positional embeddings of any length.

Convolution module starts with a pointwise convolution and a gated linear unit (GLU). This is followed by a single 1-D depthwise convolution

Convolution module starts with a pointwise convolution and a gated linear unit (GLU). This is followed by a single 1-D depthwise convolution

When groups == in_channels and out_channels == K * in_channels, where K is a positive integer, this operation is termed in literature as dept

When groups == in_channels and out_channels == K * in_channels, where K is a positive integer, this operation is termed in literature as dept

Multi-headed attention from 'Attention Is All You Need' paper

Args: d_model (int): The dimension of model num_heads (int): The number of attention heads. dropout_p (float): probabilit

Relative positional encoding module. Args: d_model: Embedding dimension. max_len: Maximum input length.

Same positional encoding method as NeMo library

Multi-head attention with relative positional encoding. This concept was proposed in the "Transformer-XL: Attentive Language Models Beyond a

SqueezeformerBlock is a simpler block structure similar to the standard Transformer block, where the MHA and convolution modules are each dir

SqueezeformerBlock is a simpler block structure similar to the standard Transformer block, where the MHA and convolution modules are each dir

Squeezeformer encoder first processes the input with a convolution subsampling layer and then with a number of squeezeformer blocks. Arg

Squeezeformer encoder first processes the input with a convolution subsampling layer and then with a number of squeezeformer blocks. Arg

Convolutional subsampler: a stack of 1D convolution (along temporal dimension) followed by non-linear activation via gated linear units (http

Multi-headed attention from 'Attention Is All You Need' paper

When groups == in_channels and out_channels == K * in_channels, where K is a positive integer, this operation is termed in literature as dept

When groups == in_channels and out_channels == K * in_channels, where K is a positive integer, this operation is termed in literature as dept

stretches/ squeezes input over time dimension Args: rate (float,float): lower and upper amount of resampling rate. Should both b

stretches/ squeezes input over time dimension Args: landmarks : xyz_landmarks .. array of strings n_fingers : num finger

stretches/ squeezes input over time dimension Args: landmarks : xyz_landmarks .. array of strings n_fingers : num finger

stretches/ squeezes input over time dimension Args: landmarks : xyz_landmarks .. array of strings n_fingers : num finger

Select one of transforms to apply. Selected transform will be called with `force_apply=True`. Transforms probabilities will be normalized to one 1

Select one of transforms to apply. Selected transform will be called with `force_apply=True`. Transforms probabilities will be normalized to one 1

Select one of transforms to apply. Selected transform will be called with `force_apply=True`. Transforms probabilities will be normalized to one 1

stretches/ squeezes input over time dimension Args: landmarks : xyz_landmarks .. array of strings n_fingers : num finger

stretches/ squeezes input over time dimension Args: landmarks : xyz_landmarks .. array of strings n_fingers : num finger

Relative positional encoding module. Args: d_model: Embedding dimension. max_len: Maximum input length.

stretches/ squeezes input over time dimension Args: rate (float,float): lower and upper amount of resampling rate. Should both b

stretches/ squeezes input over time dimension Args: scale (float, float) or None rate (f

stretches/ squeezes input over time dimension Args: scale (float, float) or None rate (f

stretches/ squeezes input over time dimension Args: rate (float,float): lower and upper amount of resampling rate. Should both b

stretches/ squeezes input over time dimension Args: rate (float,float): lower and upper amount of resampling rate. Should both b

stretches/ squeezes input over time dimension Args: rate (float,float): lower and upper amount of resampling rate. Should both b

stretches/ squeezes input over time dimension Args: rate (float,float): lower and upper amount of resampling rate. Should both b

stretches/ squeezes input over time dimension Args: rate (float,float): lower and upper amount of resampling rate. Should both b

stretches/ squeezes input over time dimension Args: rate (float,float): lower and upper amount of resampling rate. Should both b

stretches/ squeezes input over time dimension Args: rate (float,float): lower and upper amount of resampling rate. Should both b

stretches/ squeezes input over time dimension Args: rate (float,float): lower and upper amount of resampling rate. Should both b

stretches/ squeezes input over time dimension Args: rate (float,float): lower and upper amount of resampling rate. Should both b

stretches/ squeezes input over time dimension Args: rate (float,float): lower and upper amount of resampling rate. Should both b