Embeds a given tensor. Args: inputs: A `Tensor` with type `int32` or `int64` containing the ids to be looked up in `lookup table`.

memory: encoder outputs. (N, T1, d_model) src_masks: (N, T1) Returns logits: (N, T2, V). float32. y_hat: (N,

Gets training / evaluation mini-batches fpath1: source file path. string. fpath2: target file path. string. maxlen1: source sent maximu

Applies multihead attention. See 3.2.2 queries: A 3d tensor with shape of [N, T_q, d_model]. keys: A 3d tensor with shape of [N, T_k, d_model]

Applies multihead attention. Args: queries: A 3d tensor with shape of [N, T_q, C_q]. keys: A 3d tensor with shape of [N, T_k, C_k

Calculates bleu score and appends the report to translation ref: reference file path translation: model output file path Returns tran

Converts string to number. Used for `generator_fn`. inp: 1d byte array. type: "x" (source side) or "y" (target side) dict: token2idx di

Returns memory: encoder outputs. (N, T1, d_model)

Predicts autoregressively At inference, input ys is ignored. Returns y_hat: (N, T2)

Point-wise feed forward net. Args: inputs: A 3d tensor with shape of [N, T, C]. num_units: A list of two integers. scope: O

position-wise feed forward net. See 3.3 inputs: A 3d tensor with shape of [N, T, C]. num_units: A list of two integers. scope: Option

Gets hypotheses. num_batches: scalar. num_samples: scalar. sess: tensorflow sess object tensor: target tensor to fetch dict: idx2t

Applies layer normalization. See https://arxiv.org/abs/1607.06450. inputs: A tensor with 2 or more dimensions, where the first dimension has `batc

Loads vocabulary file and returns idx<->token maps vocab_fpath: string. vocabulary file path. Note that these are reserved 0: <pad>, 1:

Constructs vocabulary. Args: fpath: A string. Input file path. fname: A string. Output file name. Writes vocabular

Masks paddings on keys or queries to inputs inputs: 3d tensor. (h*N, T_q, T_k) key_masks: 3d tensor. (N, 1, T_k) type: string. "key" | "fu

Applies layer normalization. Args: inputs: A tensor with 2 or more dimensions, where the first dimension has `batch_size`.

Sinusoidal Positional_Encoding. See 3.5 inputs: 3d tensor. (N, T, E) maxlen: scalar. Must be >= T masking: Boolean. If True, padding posit

Sinusoidal Positional_Encoding. Args: inputs: A 2d Tensor with shape of (N, T). num_units: Output dimensionality zero_pad: Bool

Gets size of tensor shape shp: TensorShape Returns size

Calculates the number of batches. total_num: total sample number batch_size Returns number of batches, allowing for remainders.

Converts int32 tensor to string tensor. inputs: 1d int32 tensor. indices. idx2token: dictionary Returns 1d string tensor.

Constructs token embedding matrix. Note that the column of index 0's are set to zeros. vocab_size: scalar. V. num_units: embedding dimensi

Batchify data sents1: list of source sents sents2: list of target sents vocab_fpath: string. vocabulary file path. batch_size: sca

Applies label smoothing. See 5.4 and https://arxiv.org/abs/1512.00567. inputs: 3d tensor. [N, T, V], where V is the number of vocabulary. epsi

Applies label smoothing. See https://arxiv.org/abs/1512.00567. Args: inputs: A 3d tensor with shape of [N, T, V], where V is the number

Loads source and target data and filters out too lengthy samples. fpath1: source file path. string. fpath2: target file path. string. m

Loads hparams and overrides parser parser: argsparse parser path: directory or file where hparams are saved

Noam scheme learning rate decay init_lr: initial learning rate. scalar. global_step: scalar. warmup_steps: scalar. During warmup_steps, le

Processes translation outputs. hypotheses: list of encoded predictions idx2token: dictionary Returns processed hypotheses

Load raw data -> Preprocessing -> Segmenting with sentencepice hp: hyperparams. argparse.

Saves hparams to path hparams: argsparse object. path: output directory. Writes hparams as literal dictionary to path.

Saves information about variables such as their name, shape, and total parameter number fpath: string. output file path Writes a text

See 3.2.1. Q: Packed queries. 3d tensor. [N, T_q, d_k]. K: Packed keys. 3d tensor. [N, T_k, d_k]. V: Packed values. 3d tensor. [N, T_k, d_

Returns loss: scalar. train_op: training operation global_step: scalar. summaries: training summary node

Generates training / evaluation data sents1: list of source sents sents2: list of target sents vocab_fpath: string. vocabulary file pat