Break a Dataset of tokens into blocks. Args: dataset (~torch.utils.data.Dataset): dataset to break into blocks sizes (List[int]):

A pair of torch.utils.data.Datasets. Args: src (torch.utils.data.Dataset): source dataset to wrap src_sizes (List[int]): sou

Multi-headed attention. See "Attention Is All You Need" for more details.

A simple running average, for things like FPS (frames per second) counters.

Records progress through the set of constraints for each item in the beam using a trie.

A mapping from symbols to consecutive integers

Records progress through the set of linear nonbranching constraints with gaps.

Randomly samples from a given dataset at each epoch. Sampling is done with or without replacement, depending on the "replace" parameter.

This is an implementation of the efficient softmax approximation for graphical processing units (GPU), described in the paper "Efficient soft

A LayerDrop implementation based on :class:`torch.nn.ModuleList`. We refresh the choice of which layers to drop every time we iterate ov

A sorted dictionary of :class:`Meters`. Meters are sorted according to a priority that is given when the meter is first added to the dictiona

Wrap a DistributedDataParallel module and forward requests for missing attributes to the module wrapped by DDP (the twice-wrapped module).

Piecewise linear function. Can be configured with a string.

A simple PyTorch Hub interface to RoBERTa. Usage: https://github.com/pytorch/fairseq/tree/main/examples/roberta

Zip multiple :class:`~fairseq.data.FairseqDataset` instances together. Shorter datasets are repeated in a round-robin fashion to match the length

Truncate a sequence by returning the first truncation_length tokens

Keep track of what's going on in the binarizer

contextmanager to read a chunck of a file line by line.

Wrapper around an iterable that maintains the iteration count. Args: iterable (iterable): iterable to wrap start (int): starting

This module learns positional embeddings up to a fixed maximum size. Padding ids are ignored by either offsetting based on padding_idx or

NoOp simply passes the input as the output.

A sharded wrapper around an iterable, padded to length. Args: iterable (iterable): iterable to wrap num_shards (int): number of s

A :class:`~fairseq.data.FairseqDataset` wrapper that transform bos on collated samples of language pair dataset. Note that the transformation

Transformer encoder without token embeddings.

A fixed-sized Dictionary that operates on integer-valued tokens wth a trivial (identity) token <-> id mapping. Special symbols (bos, eos,

r"""Implements Adam algorithm. This implementation is modified from torch.optim.Adam based on: `Fixed Weight Decay Regularization in Adam`

Bucket and pad item lengths to the nearest bucket size. This can be used to reduce the number of unique batch shapes, which is important on T

Encoder layer for Wav2vec2 encoder

Represents a node in a trie managing unordered constraints.

Convolutional subsampler: a stack of 1D convolution (along temporal dimension) followed by non-linear activation via gated linear units (https

Convolutional subsampler: a stack of 2D convolution based on ESPnet implementation (https://github.com/espnet/espnet) Args: input_cha

A wrapper around TokenBlockDataset for BART dataset. Args: dataset (TokenBlockDataset): dataset to wrap sizes (List[int]): s

Selects every nth element, where n is the index

Multi-headed attention with Gating and Downsampling