Residual Vector Quantizer. Args: dimension (int): Dimension of the codebooks. n_q (int): Number of residual vector quantizers used

r"""Adaptive Layer Normalization

Conv1d + GLU(Gated Linear Unit) with residual connection. For GLU refer to https://arxiv.org/abs/1612.08083 paper.

ConvNeXt Block adapted from https://github.com/facebookresearch/ConvNeXt to 1D audio signal. Args: dim (int): Number of input channels.

Implements Warmup learning rate schedule until 'warmup_steps', going from 'init_lr' to 'peak_lr' for multiple optimizers.

Modifies the backpropped derivatives of a function to try to encourage, for each channel, that it is positive at least a proportion `threshol

Dialted and Depth-Separable Convolution

Codebook with Euclidean distance. Args: dim (int): Dimension. codebook_size (int): Codebook size. kmeans_init (bool): Whet

MelStyleEncoder

Multi-Head Attention module

r"""Allows the model to jointly attend to information from different representation subspaces as described in the paper: `Attention Is All You

Residual vector quantization implementation. Follows Algorithm 1. in https://arxiv.org/pdf/2107.03312.pdf

Implements 'Scaled Adam', a variant of Adam where we scale each parameter's update proportional to the norm of that parameter; and also lea

Scaled Dot-Product Attention

Synthesizer for Training

Synthesizer for Training

r"""TransformerEncoder is a stack of N encoder layers. Users can build the BERT(https://arxiv.org/abs/1810.04805) model with corresponding paramet

Vector quantization implementation. Currently supports only euclidean distance. Args: dim (int): Dimension codebook_size (int)

AMPBlock applies Snake / SnakeBeta activation functions with trainable parameters that control periodicity, defined for each layer. AMPBlock1

AMPBlock applies Snake / SnakeBeta activation functions with trainable parameters that control periodicity, defined for each layer. Unlike AM

Attentive statistics pooling: Channel- and context-dependent statistics pooling, first used in ECAPA_TDNN.

This class adds to class Optimizer the capability to optimize parameters in batches: it will stack the parameters and their grads for you so

BigVGAN is a neural vocoder model that applies anti-aliased periodic activation for residual blocks (resblocks). New in BigVGAN-v2: it can op