Restores model and optimizer from given directory. Specify what shoud be restored Returns: Last training step for the

Remove entries that are None or [] from an iterable.

Remove entries that are None or [] from an iterable.

force one of token_ids to be generated by setting prob of all other tokens to 0

Prediction/evaluation loop, shared by `evaluate()` and `predict()`. Works both with or without labels. NOTE: One issue is o

Prediction/evaluation loop, shared by `evaluate()` and `predict()`. Works both with or without labels. NOTE: One issue is o

Add a new hypothesis to the list.

Compute 'Scaled Dot Product Attention

Compute 'Scaled Dot Product Attention

Returns how many items in batch have length >= i at step i. Examples: [5] -> [1, 1, 1, 1, 1] [5, 5] -> [2, 2, 2, 2, 2] [5, 3] ->

Copied from fairseq for no_repeat_ngram in beam_search

Copied from fairseq for no_repeat_ngram in beam_search

``indices`` of size ``(batch_size, d_1, ..., d_n)`` indexes into dimension 2 of a target tensor, which has size ``(batch_size, sequence_length, embe

Includes several features from "Jointly Learning to Align and Translate with Transformer Models" (Garg et al., EMNLP 2019).

:returns next idx, column id

If it's bert, we also cache the normalized version of question/column/table for schema linking

Serializes this instance to a JSON string.