NICE Flow

Linear with weight normalization

Global Attention between encoder and decoder

Multi-head Attention

This module produces sinusoidal positional embeddings of any length. Padding symbols are ignored.

Conv1d with weight normalization

Cross Entropy loss with label smoothing. For training, the loss is smoothed with parameter eps, while for evaluation, the smoothing is disabl

NMT Flow Block

One Unit of NMTFlowStep

r"""Implements AdamW algorithm. This implementation is modified from torch.optim.Adam based on: `Fixed Weight Decay Regularization in Adam`

A class that listens for exceptions in children processes and propagates the tracebacks to the parent process.

Set the learning rate of each parameter group to the initial lr decayed by gamma every epoch. When last_epoch=-1, sets initial lr as lr. We al

NMT model with Generative Flow.

core module for flow nmt model

Decay the LR based on the inverse square root of the update number. We also support a warmup phase where we linearly increase the learning ra

NMT Flow

Unit for POS Attention

One step of NMTFlow

class for Prior with a NMTFlow inside

Decoder with Recurrent Neural Networks

Src Encoder to encode source sentence with Recurrent Neural Networks

Posterior with Recurrent Neural Networks

Posterior with Recurrent Neural Networks

Simple Decoder to predict translations from latent z

Decoder with Transformer

Src Encoder to encode source sentence with Transformer

Posterior with Transformer

Implements data parallelism at the flow level.

Decoder to predict translations from latent z

Src Encoder to encode source sentence

Normalizing Flow base class

Length Predictor

posterior class