Context gate is a decoder module that takes as input the previous word embedding, the current decoder state and the attention state, and

r""" Global attention takes a matrix and a query vector. It then computes a parameterized convex combination of the matrix based on the in

Multi-Head Attention module from "Attention is All You Need" :cite:`DBLP:journals/corr/VaswaniSPUJGKP17`. Similar to standard `dot` attention

A two-layer Feed-Forward-Network with residual layer norm. Args: d_model (int): the size of input for the first-layer of the FFN.

Wrapper around multiple decoder final hidden states.

Gated convolution for CNN class

Sinusoidal positional encoding for non-recurrent neural networks. Implementation based on "Attention Is All You Need" :cite:`DBLP:journals/co

Container for subfields. Text data might use POS/NER/etc labels in addition to tokens. This class associates the "base" :class:`Field` with a

Parse ['<src>-<tgt>', ...] into ['<src>','<tgt>', ...]

Defines an audio datatype and instructions for converting to Tensor. See :class:`Fields` for attribute descriptions.

Average Attention module from "Accelerating Neural Transformer via an Average Attention Network" :cite:`DBLP:journals/corr/abs-1805-00631

Generation beam search. Note that the attributes list is not exhaustive. Rather, it highlights tensors to document their shape. (Since the st

Basic layer that casts its input to a specific data type. The same tensor is returned if the data type is already correct.

Conv attention takes a key matrix, a value matrix and a query vector. Attention weight is calculated by key matrix with the query vector

An implementation of pointer-generator networks :cite:`DBLP:journals/corr/SeeLM17`. These networks consider copying words directly from t

Yield data from sharded dataset files. Args: dataset_paths: a list containing the locations of dataset files. fields (dict[str, F

A simple network container. Parameters are a list of modules. Inputs are a 3d Tensor whose last dimension is the same length as the l

Words embeddings for encoder/decoder. Additionally includes ability to add sparse input features based on "Linguistic Input Features Improve

Dummy Decoder that delegates to individual real Decoders.

Dummy Encoder that delegates to individual real Encoders.

Dummy Generator that delegates to individual real Generators, and then averages the resulting target distributions.

Dummy NMTModel wrapping individual real NMTModels.

A class that listens for exceptions in children processes and propagates the tracebacks to the parent process.

Extract feature vectors from a batch of frames.

Implements Adam algorithm. Currently GPU-only. Requires Apex to be installed via ``python setup.py install --cuda_ext --cpp_ext``. It h

Select next tokens randomly from the top k possible next tokens. The ``scores`` attribute's lists are the score, after applying temperature,

With label smoothing, KL-divergence between q_{smoothed ground truth prob.}(w) and p_{prob. computed by model}(w) is minimized.

Simple model saver to filesystem

This takes a list of iterable objects (DatasetLazyIter) and their respective weights, and yields a batch in the wanted proportions.

Implement multiple optimizers needed for sparse adam

Standard NMT Loss Computation.

Turn batches of feature vectors into sequences for each video. Assumes data is ordered (use one reconstructor per process). :func:`push()` bat

Wrap a model with server functionality. Args: opt (dict): Options for the Translator model_id (int): Model ID preprocess_

An implementation of sparsemax loss, first proposed in :cite:`DBLP:journals/corr/MartinsA16`. If using a sparse output layer, it is not p

Stacked CNN class

Args: d_model (int): the dimension of keys/values/queries in :class:`MultiHeadedAttention`, also the input size of the

A single layer of the transformer encoder. Args: d_model (int): the dimension of keys/values/queries in MultiHead

Container for a translated sentence. Attributes: src (LongTensor): Source word IDs. src_raw (List[str]): Raw source words.

Defines an vector datatype and instructions for converting to Tensor. See :class:`Fields` for attribute descriptions.

For each video, yield its frames.

Read audio data from disk. Args: sample_rate (int): sample_rate. window_size (float) : window size for spectrogram in seconds.

A simple encoder CNN -> RNN for audio input. Args: rnn_type (str): Type of RNN (e.g. GRU, LSTM, etc). enc_layers (int): Number of

Apply the context gate to both contexts

Decoder based on "Convolutional Sequence to Sequence Learning" :cite:`DBLP:journals/corr/GehringAGYD17`. Consists of residual convolutional l

Encoder based on "Convolutional Sequence to Sequence Learning" :cite:`DBLP:journals/corr/GehringAGYD17`.

Copy generator criterion.

Copy Generator Loss Computation.

Read data from file system and yield as dicts. Raises: onmt.inputters.datareader_base.MissingDependencyException: A number of

Contain data and process it. A dataset is an object that accepts sequences of raw data (sentence pairs in the case of machine translation) an

Base class for generation strategies. Args: pad (int): Magic integer in output vocab. bos (int): Magic integer in output vocab.

Abstract class for decoders. Args: attentional (bool): The decoder returns non-empty attention.

Deprecate action

Base encoder class. Specifies the interface used by different encoder types and required by :class:`onmt.Models.NMTModel`. .. mermaid::

NMT re-ranking. Args: alpha (float): Length parameter. beta (float): Coverage parameter. length_penalty (str): Length penal

Read image data from disk. Args: truncate (tuple[int] or NoneType): maximum img size. Use ``(0,0)`` or ``None`` for unlimited

A simple encoder CNN -> RNN for image src. Args: num_layers (int): number of encoder layers. bidirectional (bool): bidirectional

Input feeding based decoder. See :class:`~onmt.decoders.decoder.RNNDecoderBase` for options. Based around the input feeding approach from

Class for managing efficient loss computation. Handles sharding next step predictions and accumulating multiple loss computations Us

Implementation of the matrix-tree theorem for computing marginals of non-projective dependency parsing. This attention layer is used in the pa

A trivial non-recurrent encoder. Simply applies mean pooling. Args: num_layers (int): number of replicated layers embeddings (onmt.

Base class for model saving operations Inherited classes must implement private methods: * `_save` * `_rm_checkpoint

Core trainable object in OpenNMT. Implements a trainable interface for a simple, generic encoder + decoder model. Args: encoder (o

Controller class for optimization. Mostly a thin wrapper for `optim`, but also useful for implementing rate scheduling beyond what is cur

Returns the Length and Coverage Penalty function for Beam Search. Args: length_pen (str): option name of length pen cov_pen (str)

Base recurrent attention-based decoder class. Specifies the interface used by different decoder types and required by :class:`~onmt.models.NM

A generic recurrent neural network encoder. Args: rnn_type (str): style of recurrent unit to use, one of [RNN, LSTM, GRU, SRU]

Report Manager Base class Inherited classes should override: * `_report_training` * `_report_step`

Implementation of "Training RNNs as Fast as CNNs" :cite:`DBLP:journals/corr/abs-1709-02755` TODO: turn to pytorch's implementation when

Apply the context gate only to the source context