Sanitized serialization to use with TensorBoard’s hparams

Filter a distribution of logits using top-k and/or nucleus (top-p) filtering Args: logits: logits distribution shape (batch size,

Filter a distribution of logits using top-k and/or nucleus (top-p) filtering Args: logits: logits distribution shape (batch size,

Initialize n-best list of hypotheses.

Initialize n-best list of hypotheses.

force one of token_ids to be generated by setting prob of all other tokens to 0

force one of token_ids to be generated by setting prob of all other tokens to 0

Generate sequences for each example with beam search.

Generate sequences for each example with beam search.

Generate sequences for each example without beam search (num_beams == 1). All returned sequence are generated independantly.

Generate sequences for each example without beam search (num_beams == 1). All returned sequence are generated independantly.

Identical to the XLM create_sinusoidal_embeddings except features are not interleaved. The cos features are in the 2nd half of the vector.

Identical to the XLM create_sinusoidal_embeddings except features are not interleaved. The cos features are in the 2nd half of the vector.

Prepare masks that ignore padding tokens in the decoder and a causal mask for the decoder if none are provided. This mimics the default behavior i

Prepare masks that ignore padding tokens in the decoder and a causal mask for the decoder if none are provided. This mimics the default behavior i

Setup the optional Weights & Biases (`wandb`) integration. One can override this method to customize the setup if needed. Find more

Setup the optional Weights & Biases (`wandb`) integration. One can override this method to customize the setup if needed. Find more

Compute 'Scaled Dot Product Attention

Compute 'Scaled Dot Product Attention

Replace non-padding symbols with their position numbers. Position numbers begin at padding_idx+1. Padding symbols are ignored. This is modified f

Replace non-padding symbols with their position numbers. Position numbers begin at padding_idx+1. Padding symbols are ignored. This is modified f

read table and column info

Since bert cannot handle sequence longer than 512, each column/table is encoded individually The representation of a column/table is