A mapping from symbols to consecutive integers

A pair of torch.utils.data.Datasets. Args: src (torch.utils.data.Dataset): source dataset to wrap src_sizes (List[int]): sou

Break a Dataset of tokens into blocks. Args: dataset (~torch.utils.data.Dataset): dataset to break into blocks sizes (List[int]):

Multi-headed attention. See "Attention Is All You Need" for more details.

Records progress through the set of constraints for each item in the beam using a trie.

VGG + Transformer encoder

This is an implementation of the efficient softmax approximation for graphical processing units (GPU), described in the paper "Efficient soft

1D convolution over an input of shape (time x batch x channel) The implementation uses gemm to perform the convolution. This implementation i

Records progress through the set of linear nonbranching constraints with gaps.

A sorted dictionary of :class:`Meters`. Meters are sorted according to a priority that is given when the meter is first added to the dictiona

Model parallel Multi-headed attention. This performs the Multi-headed attention over multiple gpus. See "Megatron-LM: https://arxiv.org/pdf/1

Randomly samples from a given dataset at each epoch. Sampling is done with or without replacement, depending on the "replace" parameter.

Scores the target for a given source sentence.

Computes the sum/avg duration of some event in seconds

Computes the average occurrence of some event per second

Decoder layer block. In the original paper each operation (multi-head attention, encoder attention or FFN) is postprocessed with: `dropout ->

Truncate a sequence by returning the first truncation_length tokens

A LayerDrop implementation based on :class:`torch.nn.ModuleList`. We refresh the choice of which layers to drop every time we iterate ov

This module learns positional embeddings up to a fixed maximum size. Padding ids are ignored by either offsetting based on padding_idx or

An optimized version of nn.Conv1d. At training time, this module uses ConvTBC, which is an optimized version of Conv1d. At inference time, it

Stores multiple instances of FairseqDataset together and in every iteration creates a batch by first sampling a dataset according to a specif

A simple PyTorch Hub interface to RoBERTa. Usage: https://github.com/pytorch/fairseq/tree/master/examples/roberta

Zip multiple :class:`~fairseq.data.FairseqDataset` instances together. Shorter datasets are repeated in a round-robin fashion to match the length

A sharded wrapper around an iterable, padded to length. Args: iterable (iterable): iterable to wrap num_shards (int): number of s

Sparse Multi-Headed Attention. "Generating Long Sequences with Sparse Transformers". Implements fixed factorized self attention, where l=str

A :class:`~fairseq.data.FairseqDataset` wrapper that appends/prepends/strips EOS. Note that the transformation is applied in :func:`collater`.

Transformer decoder consisting of *args.decoder_layers* layers. Each layer is a :class:`TransformerDecoderLayer`. Args: args (ar

Decoder layer block. In the original paper each operation (multi-head attention, encoder attention or FFN) is postprocessed with: `dropout ->

Encoder layer block. In the original paper each operation (multi-head attention or FFN) is postprocessed with: `dropout -> add residual -> la

Implements Adam algorithm. This implementation is modified from torch.optim.Adam based on: `Fixed Weight Decay Regularization in Adam` (s

Function to calculate Average Lagging from STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Pref

Function to calculate Average Proportion from Can neural machine translation do simultaneous translation? (https://arxiv.org/abs/1606.020

Sets up a backtranslation dataset which takes a tgt batch, generates a src using a tgt-src backtranslation function (*backtranslation_fn*),

Bucket and pad item lengths to the nearest bucket size. This can be used to reduce the number of unique batch shapes, which is important on T

Represents a node in a trie managing unordered constraints.

Wrapper around an iterable that maintains the iteration count. Args: iterable (iterable): iterable to wrap start (int): starting

A wrapper around TokenBlockDataset for BART dataset. Args: dataset (TokenBlockDataset): dataset to wrap sizes (List[int]): s

Function to calculate Differentiable Average Lagging from Monotonic Infinite Lookback Attention for Simultaneous Machine Translation (htt

Selects every nth element, where n is the index

Multi-headed attention with Gating and Downsampling

A wrapper around an ensemble of models.

Convolutional decoder

Loader for TorchNet IndexedDataset

LSTM decoder.

LightConv decoder consisting of *args.decoder_layers* layers. Each layer is a :class:`LightConvDecoderLayer`. Args: args (argpar

A wrapper Dataset for masked language modelling. The dataset wraps around TokenBlockDataset or BlockedPairDataset and creates a batch whe

Model Parallel Transformer decoder consisting of *args.decoder_layers* layers. Each layer is a :class:`ModelParallelTransformerDecoderLayer`.

Source: https://stackoverflow.com/a/54128391

Implements collate function mainly for seq2seq tasks This expects each sample to contain feature (src_tokens) and targets.

Single-head attention that supports Gating and Downsampling

A :class:`~fairseq.data.FairseqDataset` wrapper that transform bos on collated samples of language pair dataset. Note that the transformation

Transformer decoder consisting of *args.decoder_layers* layers. Each layer is a :class:`TransformerDecoderLayer`. Args: args (arg

Transformer encoder consisting of *args.encoder_layers* layers. Each layer is a :class:`TransformerEncoderLayer`. Args: args (ar

Implementation for a Bi-directional Transformer based Sentence Encoder used in BERT/XLM style pre-trained models. This first computes th

Fake scalar quantization of the activations using a forward hook. Args: - module. a nn.Module for which we quantize the *post-activa

Implements Adafactor algorithm. This implementation is based on: `Adafactor: Adaptive Learning Rates with Sublinear Memory Cost` (see htt

Implements Adamax algorithm (a variant of Adam based on infinity norm). It has been proposed in `Adam: A Method for Stochastic Optimization`__.