Creates position encodings. Args: positions: The positions to encode of shape :math:`[B, ...]`. depth: The encoding depth

Creates position encodings. Args: positions: The positions to encode of shape :math:`[B, ...]`. depth: The encoding depth

Creates position encodings. Args: positions: The positions to encode of shape :math:`[B, ...]`. depth: The encoding depth

Returns true if all substrings appear in the haystack in order

Single forward step. Update the allocate memory flag so only the first time the memory is allocated.

Get the n-best logits from a list.

Get the n-best logits from a list.

Check if a tensor is not none and is cuda.

Make a viewless tensor. View tensors have the undesirable side-affect of retaining a reference to the originally-viewed tensor, even after ma

Use multi-tensor-applier to copy values from one list to another. We don't have a blfoat16 implementation so for now if the overflow_buf is no

Preprocess the dataset into a list of (text, label) tuples.

Given a `doc`, flatten it out so that each JSON blob contains exactly one question and one answer. Logic taken from the reference impl

Provide function that calculates accuracies.

Adds a pointer offset in units of element

Adds a pointer offset in units of element

Adds a pointer offset in units of element

Adds a pointer offset in units of Element

Adds a pointer offset in units of Element

Adds a pointer offset in units of Element

Adds a pointer offset in units of Element

Adds a tile offset

:returns: {str: [metric_score] -> float} A dictionary where keys are the names of submetrics and values are functions

See base class.

See base class.

Returns a reference to the element at a given Coord

Performs multi-headed attention from `from_tensor` to `to_tensor`. This is an implementation of multi-headed attention based on "Attention is all

Helper method to run backward step with model split into chunks (run set_virtual_pipeline_model_parallel_rank() before calling backwar

a matrix [8, 128] rowmajor * b matrix [128, 8] colmajor | b matrix [8, 128] rowmajor

a matrix [8, 128] rowmajor * b matrix [128, 8] colmajor | b matrix [8, 128] rowmajor

a matrix [8, 128] rowmajor * b matrix [128, 8] colmajor | b matrix [8, 128] rowmajor

Broadcast a list of values with a given type.

Helper function to select and build dataset.

Args: opt: the option in current environment. text_field(TextMultiField): word and feats field. for_encoder(bool): build