Universal Transformer main model_fn. Args: features: Map of features to the model. Should contain the following: "inputs":

Universal Transformer main model_fn. Args: features: Map of features to the model. Should contain the following: "inputs": Tr

Calculate Chebyshev polynomials up to order k. Return a list of sparse matrices (tuple representation).

Tokenization/string cleaning for all datasets except for SST. Original taken from https://github.com/yoonkim/CNN_sentence/blob/master/process

Tokenization/string cleaning for the SST dataset

Construct feed dictionary.

Read Word Vectors

Row-normalize sparse matrix

All ones.

Parse index file.

Create mask.

Convert a scipy sparse matrix to a torch sparse tensor.

Applies multihead attention function which is parametrised for decoding. Args: x: input (decoder input) hparams: model hyper-parameters

Applies a feed-forward function which is parametrised for decoding. Args: x: input hparams: model hyper-parameters nonpadding_mask

Applies multihead attention function which is parametrised for encoding. Args: x: input hparams: model hyper-parameters encoder_se

Applies a feed-forward function which is parametrised for encoding. Args: x: input hparams: model hyper-parameters nonpadding_mask

Uniform init.

Range of hyperparameters.

Basic Universal Transformer. This model is pretty similar to the vanilla transformer in which weights are shared between layers. For some task

Universal Transformer decoder function. Prepares all the arguments and the inputs and passes it to a core_universal_transformer_layer to decod

universal_transformer with depth-wise attention. It uses an attention mechanism-flipped vertically- over all the states from previous steps to

Universal Transformer encoder function. Prepares all the arguments and the inputs and passes it to a universal_transformer_layer to encode the

Universal Transformer with highway connection. It transforms the state using a block contaaining sel-attention and transition function and

Universal Transformer with highway connection. It transforms the state using attention and ffn and wrap this transformation with a skip-all

Universal Transformer which uses a gru as transition function. It's kind of like having a gru, filliped vertically next to the Universal Trans

Universal Transformer which uses a lstm as transition function. It's kind of like having a lstm, filliped vertically next to the Universal Tra

implements act (position-wise halting). Args: state: 3-D Tensor: [batch_size, length, channel] step: indicates number of steps ta