Validates the input pipeline. Perform some sanity checks to log user friendly information. We should error out to give users better error mes

Validates that the features and labels for the model function are valid. A valid features/labels object is the one with: - Type: A tensor or

Validate graph before running distributed training. Raises: ValueError: If the graph seems invalid for running on device

Validates the estimator_spec.

Validates TPUEstimatorSpec.predictions dict.

Wraps the ops generated by `op_fn` in tf.while_loop.

Core absolute positional attention operations.

See base class.

See base class.

See base class.

See base class.

See base class.

Returns the signals as Python dict.

Creates an equivalent `EstimatorSpec` used by CPU train/eval.

Checks that all the tensors in the graph are reachable from outputs.

Checks that all the tensors in the graph are reachable from outputs.

Checks that all the tensors in the graph are reachable from outputs.

Checks that all the tensors in the graph are reachable from outputs.

Checks that all the tensors in the graph are reachable from outputs.

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python i

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python intege

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python i

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python i

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python i

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python i

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python i

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python i

Performs multi-headed attention from `from_tensor` to `to_tensor`. This is an implementation of multi-headed attention based on "Attention is all

Performs multi-headed attention from `from_tensor` to `to_tensor`. This is an implementation of multi-headed attention based on "Attention is all

Performs multi-headed attention from `from_tensor` to `to_tensor`. This is an implementation of multi-headed attention based on "Attention is all

Performs multi-headed attention from `from_tensor` to `to_tensor`. This is an implementation of multi-headed attention based on "Attention is all

Performs multi-headed attention from `from_tensor` to `to_tensor`. This is an implementation of multi-headed attention based on "Attention is all

Performs multi-headed attention from `from_tensor` to `to_tensor`. This is an implementation of multi-headed attention based on "Attention is all

A map of modules from TF to PyTorch. This time I use a map to keep the PyTorch model as identical to the original PyTorch model as possible.

A map of modules from TF to PyTorch. I use a map to keep the PyTorch model as identical to the original PyTorch model as possible.

Returns list of utf-8 byte and a mapping to unicode strings. We specifically avoids mapping to whitespace/control characters the bpe code bar

cache hidden states into memory.

Clean up a list of simple English tokenization artifacts like spaces before punctuations and abreviated forms.

Converts a single `InputExample` into a single `InputFeatures`.

Converts a single `InputExample` into a single `InputFeatures`.

Converts a single `InputExample` into a single `InputFeatures`.

Converts a single `InputExample` into a single `InputFeatures`.

Converts a single `InputExample` into a single `InputFeatures`.

Converts a single `InputExample` into a single `InputFeatures`.

Converts a single `InputExample` into a single `InputFeatures`.

Converts a single `InputExample` into a single `InputFeatures`.

Converts a single `InputExample` into a single `InputFeatures`.

Loads a data file into a list of `InputBatch`s.

Loads a data file into a list of `InputBatch`s.

Loads a data file into a list of `InputBatch`s.

Loads a data file into a list of `InputBatch`s.

Loads a data file into a list of `InputBatch`s.

Converts user provided model_fn` as a single eval step on TPU. Similar to training, the user provided `model_fn` takes input tuple (features,

Converts user provided model_fn` as a single predict step on TPU. Args: dequeue_fn: The function to retrieve inputs, features and labels, f

Converts user provided model_fn` as a single train step on TPU. The user provided `model_fn` takes input tuple (features, labels) and produce

Create 3D attention mask from a 2D tensor mask. Args: from_tensor: 2D or 3D Tensor of shape [batch_size, from_seq_length, ...]. to_mask: in

Create 3D attention mask from a 2D tensor mask. Args: from_tensor: 2D or 3D Tensor of shape [batch_size, from_seq_length, ...]. to_mask: in

Create 3D attention mask from a 2D tensor mask. Args: from_tensor: 2D or 3D Tensor of shape [batch_size, from_seq_length, ...]. to_mask: in

Create 3D attention mask from a 2D tensor mask. Args: from_tensor: 2D or 3D Tensor of shape [batch_size, from_seq_length, ...]. to_mask: in

Create 3D attention mask from a 2D tensor mask. Args: from_tensor: 2D or 3D Tensor of shape [batch_size, from_seq_length, ...]. to_mask: in

Create 3D attention mask from a 2D tensor mask. Args: from_tensor: 2D or 3D Tensor of shape [batch_size, from_seq_length, ...]. to_mask: in

Create 3D attention mask from a 2D tensor mask. Args: from_tensor: 2D or 3D Tensor of shape [batch_size, from_seq_length, ...]. to_mask: in

Creates `TrainingInstance`s for a single document.

Creates `TrainingInstance`s for a single document.

Creates `TrainingInstance`s for a single document.

Creates `TrainingInstance`s for a single document.

Creates `TrainingInstance`s for a single document.

Creates `TrainingInstance`s for a single document.