proto*/

proto*/

Add n-dimensional embedding as the depth embedding (timing signal). Adds embeddings to represent the position of the step in the recurrent tow

Add n-dimensional embedding as the position (horizontal) timing signal. Args: x: a tensor with shape [batch, length, depth] step: step

Add n-dimensional embedding as the step (vertical) timing signal. Args: x: a tensor with shape [batch, length, depth] step: step h

Decode Universal Transformer outputs from encoder representation. It is similar to "transformer.decode", but it uses "universal_transforme

Encode Universal Transformer inputs. It is similar to "transformer.encode", but it uses "universal_transformer_util.universal_transformer_

Encode transformer inputs. Args: inputs: Transformer inputs [batch_size, input_length, input_height, hidden_dim] which will be

Helper function, assigns unique layer IDs.

Provides the function that is used in universal transforemr steps. Args: x: input hparams: model hyper-parameters ffn_unit: feed-f

if smoothing == 0, it's one-hot method if 0 < smoothing < 1, it's smooth method

if smoothing == 0, it's one-hot method if 0 < smoothing < 1, it's smooth method

Loads input data from gcn/data directory ind.dataset_str.x => the feature vectors and adjacency matrix of the training instances as list;

dataset: name of dataset test_proportion: ratio of test train split seed: random seed for random splitting of dataset

dataset: name of dataset test_proportion: ratio of test train split seed: random seed for random splitting of dataset

Dropout for sparse tensors.

Convert sparse matrix to tuple representation.

A single training step

ACT based models. Implementations of all act models are based on craffel@'s cl/160711592. Args: x: input hparams: model hyper-para

The UTAct layer where the final state is the accumulation of all states. (similar to the main ACT paper: --> check the issue of differentiabili

Basic universal_transformer with ACT based on remainder-distribution ACT. Args: x: input hparams: model hyper-parameters ffn_unit:

The UTAct with global halting probability (not position-wise). Args: x: input hparams: model hyper-parameters ffn_unit: feed-forw

universal_transformer with ACT with random halting probability. Args: x: input hparams: model hyper-parameters ffn_unit: feed-forw

All zeros.