Computes and stores the average and current value

Break a Dataset of tokens into blocks. Args: dataset (~torch.utils.data.Dataset): dataset to break into blocks sizes (List[int]):

Implementation for a Bi-directional Transformer based Sentence Encoder used in BERT/XLM style pre-trained models. This first computes th

A pair of torch.utils.data.Datasets. Args: src (torch.utils.data.Dataset): source dataset to wrap src_sizes (List[int]): sou

MultiHeadAttention

Computes the average occurrence of some event per second

This is an implementation of the efficient softmax approximation for graphical processing units (GPU), described in the paper "Efficient soft

1D convolution over an input of shape (time x batch x channel) The implementation uses gemm to perform the convolution. This implementation i

Computes the sum/avg duration of some event in seconds

Transformer encoder consisting of *args.encoder_layers* layers. Each layer is a :class:`TransformerEncoderLayer`. Args: args (ar

Loader for TorchNet IndexedDataset

Infix operators. The calling sequence for the infix is: x |op| y

This module learns positional embeddings up to a fixed maximum size. Padding ids are ignored by either offsetting based on padding_idx or

Zip multiple :class:`~fairseq.data.FairseqDataset` instances together. Shorter datasets are repeated in a round-robin fashion to match the length

A sharded wrapper around an iterable, padded to length. Args: iterable (iterable): iterable to wrap num_shards (int): number of s

Transformer decoder consisting of *args.decoder_layers* layers. Each layer is a :class:`TransformerDecoderLayer`. Args: args (ar

Implements a Transformer Encoder Layer used in BERT/XLM style pre-trained models.

Implements Adam algorithm. This implementation is modified from torch.optim.Adam based on: `Fixed Weight Decay Regularization in Adam` (s

Wrapper around an iterable that maintains the iteration count. Args: iterable (iterable): iterable to wrap Attributes: count

A mapping from symbols to consecutive integers

Selects every nth element, where n is the index

Multi-headed attention with Gating and Downsampling

An extraction task on a article.

Convolutional decoder

Takes a text file as input and binarizes it in memory at instantiation. Original lines are also kept in memory

r"""Implements LAMB algorithm. It has been proposed in `Large Batch Optimization for Deep Learning: Training BERT in 76 minutes`_. Arguments:

LightConv decoder consisting of *args.decoder_layers* layers. Each layer is a :class:`LightConvDecoderLayer`. Args: args (argpar

An optimized version of nn.Conv1d. At training time, this module uses ConvTBC, which is an optimized version of Conv1d. At inference time, it

A wrapper Dataset for masked language modelling. The dataset wraps around TokenBlockDataset or BlockedPairDataset and creates a batch whe

A wrapper around torch.utils.data.Dataset for monolingual data. Args: dataset (torch.utils.data.Dataset): dataset to wrap si

Source: https://stackoverflow.com/a/54128391

Scores the target for a given source sentence.

Single-head attention that supports Gating and Downsampling

Fixed text of template

Implements Adafactor algorithm. This implementation is based on: `Adafactor: Adaptive Learning Rates with Sublinear Memory Cost` (see htt

Implements Adamax algorithm (a variant of Adam based on infinity norm). It has been proposed in `Adam: A Method for Stochastic Optimization`__.

Sets up a backtranslation dataset which takes a tgt batch, generates a src using a tgt-src backtranslation function (*backtranslation_fn*),

This module provides an optimized MM for beam decoding with attention. It leverage the fact that the source-side of the input is replicated beam

Dictionary for BERT task. This extends MaskedLMDictionary by adding support for cls and sep symbols.

Break a Dataset of tokens into sentence pair blocks for next sentence prediction as well as masked language model. High-level logics ar

A wrapper around a dictionary of :class:`FairseqEncoder` objects. We run forward on each encoder and return a dictionary of outputs. The fir

MC controller. Implements the :class:`~fairseq.models.FairseqDecoder` interface required by :class:`~fairseq.models.FairseqLanguageModel`.

Electra discriminator encoder. Implements the :class:`~fairseq.models.FairseqDecoder` interface required by :class:`~fairseq.models.FairseqLa

Head for masked language modeling.

Dynamic lightweight convolution taking T x B x C inputs Args: input_size: # of channels of the input kernel_size: convolution chan

Head for sentence-level classification tasks.

A simple PyTorch Hub interface to electra. Usage: https://github.com/pytorch/fairseq/tree/master/examples/electra

A wrapper around an ensemble of models.

A wrapper around an ensemble of models.

A wrapper around an ensemble of models.

Convolutional decoder

Convolutional encoder consisting of `len(convolutions)` layers. Args: dictionary (~fairseq.data.Dictionary): encoding dictionary

Convolutional encoder

A fully convolutional model, i.e. a convolutional encoder and a convolutional decoder, as described in `"Convolutional Sequence to Sequence

Implements Adam algorithm. Currently GPU-only. Requires Apex to be installed via ``python setup.py install --cuda_ext --cpp_ext``. It ha