Convert to numpy and return a sample consumed by the batch producer.

Builds the dot product mask. Args: sequence_length: The sequence length. num_heads: The number of heads. maximum_length: Optional size o

Build token types and paddings, trim if needed, and pad if needed.

XXX

Returns the logical capacity based on extent and layout. May differ from size().

Returns the ceiling of (a / b)

< Efficiently disables all accesses guarded by mask

< Efficiently disables all accesses guarded by mask

Clears the predicates

Clears the predicates

Clears the predicate set efficiently

Clears the predicate set efficiently

Collect min/max values Args: x_np: A numpy array to be processed. Raises: RuntimeError: when the input shape changed

The full procedural name indicates architecture, extended name, tile size

Apply conversion to val. Recursively apply conversion if `val` #is a nested tuple/list structure.

Convert and save to FT parameter Split a param into tensor_para_size if needed, and save each split param at {save_dir}/{param_name}.bin or

Converts a sequence of [tokens|ids] using the vocab.

Loads a data file into a list of `InputBatch`s.

Loads a data file into a list of `InputBatch`s.

Loads a data file into a list of `InputBatch`s.

Loads a data file into a list of `InputBatch`s.

Converts a sequence of ids in BPE tokens using the vocab.

Converts a single `InputExample` into a single `InputFeatures`.

Converts a single `InputExample` into a single `InputFeatures`.

Converts a sequence of tokens into ids using the vocab.

Creates the predictions for the masked LM objective. Note: Tokens here are vocab ids and not text tokens.

Clears the predicate set efficiently

Clears the predicate set efficiently

Ensure that numerator is divisible by the denominator.

Extent of the matrix in rows