Computes and stores the average and current value

A pair of torch.utils.data.Datasets. Args: src (torch.utils.data.Dataset): source dataset to wrap src_sizes (List[int]): sou

Break a Dataset of tokens into blocks. Args: dataset (~torch.utils.data.Dataset): dataset to break into blocks sizes (List[int]):

A mapping from symbols to consecutive integers

Multi-headed attention. See "Attention Is All You Need" for more details.

Computes the sum/avg duration of some event in seconds

Computes the average occurrence of some event per second

VGG + Transformer encoder

This is an implementation of the efficient softmax approximation for graphical processing units (GPU), described in the paper "Efficient soft

1D convolution over an input of shape (time x batch x channel) The implementation uses gemm to perform the convolution. This implementation i

Scores the target for a given source sentence.

A :class:`~fairseq.data.FairseqDataset` wrapper that appends/prepends/strips EOS. Note that the transformation is applied in :func:`collater`.

Decoder layer block. In the original paper each operation (multi-head attention, encoder attention or FFN) is postprocessed with: `dropout ->

Encoder layer block. In the original paper each operation (multi-head attention or FFN) is postprocessed with: `dropout -> add residual -> la

Loader for TorchNet IndexedDataset

This module learns positional embeddings up to a fixed maximum size. Padding ids are ignored by either offsetting based on padding_idx or

An optimized version of nn.Conv1d. At training time, this module uses ConvTBC, which is an optimized version of Conv1d. At inference time, it

Stores multiple instances of FairseqDataset together and in every iteration creates a batch by first sampling a dataset according to a specif

Zip multiple :class:`~fairseq.data.FairseqDataset` instances together. Shorter datasets are repeated in a round-robin fashion to match the length

A sharded wrapper around an iterable, padded to length. Args: iterable (iterable): iterable to wrap num_shards (int): number of s

Sparse Multi-Headed Attention. "Generating Long Sequences with Sparse Transformers". Implements fixed factorized self attention, where l=str

Transformer decoder consisting of *args.decoder_layers* layers. Each layer is a :class:`TransformerDecoderLayer`. Args: args (ar

Implements Adam algorithm. This implementation is modified from torch.optim.Adam based on: `Fixed Weight Decay Regularization in Adam` (s

Sets up a backtranslation dataset which takes a tgt batch, generates a src using a tgt-src backtranslation function (*backtranslation_fn*),

Wrapper around an iterable that maintains the iteration count. Args: iterable (iterable): iterable to wrap Attributes: count

Selects every nth element, where n is the index

Multi-headed attention with Gating and Downsampling

Dynamic lightweight convolution taking T x B x C inputs Args: input_size: # of channels of the input kernel_size: convolution chan

Convolutional decoder

Takes a text file as input and binarizes it in memory at instantiation. Original lines are also kept in memory

Wraps a MonolingualDataset and provides more context for evaluation.

LightConv decoder consisting of *args.decoder_layers* layers. Each layer is a :class:`LightConvDecoderLayer`. Args: args (argpar

Lightweight Convolution assuming the input is TxBxC Args: input_size: # of channels of the input kernel_size: convolution channels

A wrapper Dataset for masked language modelling. The dataset wraps around TokenBlockDataset or BlockedPairDataset and creates a batch whe

A wrapper around torch.utils.data.Dataset for monolingual data. Args: dataset (torch.utils.data.Dataset): dataset to wrap si

Implements collate function mainly for seq2seq tasks This expects each sample to contain feature (src_tokens) and targets.

Single-head attention that supports Gating and Downsampling

Main class for data parallel training. This class supports synchronous distributed data parallel training, where multiple workers each have a

Transformer decoder consisting of *args.decoder_layers* layers. Each layer is a :class:`TransformerDecoderLayer`. Args: args (arg

Transformer encoder consisting of *args.encoder_layers* layers. Each layer is a :class:`TransformerEncoderLayer`. Args: args (ar

Implementation for a Bi-directional Transformer based Sentence Encoder used in BERT/XLM style pre-trained models. This first computes th

Implements Adafactor algorithm. This implementation is based on: `Adafactor: Adaptive Learning Rates with Sublinear Memory Cost` (see htt

Implements Adamax algorithm (a variant of Adam based on infinity norm). It has been proposed in `Adam: A Method for Stochastic Optimization`__.

A dataset representing speech and corresponding transcription. Args: aud_paths: (List[str]): A list of str with paths to audio files

This module provides an optimized MM for beam decoding with attention. It leverage the fact that the source-side of the input is replicated beam

Dictionary for BERT task. This extends MaskedLMDictionary by adding support for cls and sep symbols.

Break a Dataset of tokens into sentence pair blocks for next sentence prediction as well as masked language model. High-level logics ar

A wrapper around a dictionary of :class:`FairseqEncoder` objects. We run forward on each encoder and return a dictionary of outputs. The fir

Helper for mapping "flat" indices to indices and offsets in an underlying dataset.

Given a model and a wav2letter++ dataset, pre-compute and store embeddings Args: input_root, str : Path to the wav2letter++

A wrapper around an ensemble of models.

Convolutional decoder

Convolutional encoder consisting of `len(convolutions)` layers. Args: dictionary (~fairseq.data.Dictionary): encoding dictionary

Convolutional encoder

A fully convolutional model, i.e. a convolutional encoder and a convolutional decoder, as described in `"Convolutional Sequence to Sequence

Implements Adam algorithm. Currently GPU-only. Requires Apex to be installed via ``python setup.py install --cuda_ext --cpp_ext``. It ha

Write features as hdf5 file in wav2letter++ compatible format

A `Highway layer <https://arxiv.org/abs/1505.00387>`_. Adopted from the AllenNLP implementation.