Creates an `input_fn` closure to be passed to TPUEstimator.

Creates an `input_fn` closure to be passed to TPUEstimator.

docs.

Constructs a `BertConfig` from a json file of parameters.

See base class.

Args: summary_type: str, "last", "first", "mean", or "attn". The method to pool the input to get a vector representation. use

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank: (opti

Creates a random int32 tensor of the shape within the vocab size.

Creates a random int32 tensor of the shape within the vocab size.

Creates a random int32 tensor of the shape within the vocab size.

Creates a random int32 tensor of the shape within the vocab size.

Creates a random int32 tensor of the shape within the vocab size.

Inserts stopping_signal into dataset via _map_fn. Here we change the data structure in the dataset, such that the return value is a dictionar

Run layer normalization on the last dimension of the tensor.

Run layer normalization on the last dimension of the tensor.

Run layer normalization on the last dimension of the tensor.

Run layer normalization on the last dimension of the tensor.

Run layer normalization on the last dimension of the tensor.

Run layer normalization on the last dimension of the tensor.

Post-attention processing.

Records the host_call structure.

Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).

Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).

Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).

Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).

Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).

Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).

Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).

Make sure we are sharing the input and output embeddings. Export to TorchScript can't handle parameter sharing so we are cloning them

Tokenizes a piece of text into its word pieces. This uses a greedy longest-match-first algorithm to perform tokenization using the given voca

Tokenizes a piece of text into its word pieces. This uses a greedy longest-match-first algorithm to perform tokenization using the given voca

Tokenizes a piece of text into its word pieces. This uses a greedy longest-match-first algorithm to perform tokenization using the given voca

Tokenizes a piece of text into its word pieces. This uses a greedy longest-match-first algorithm to perform tokenization using the given voca

Tokenizes a piece of text into its word pieces. This uses a greedy longest-match-first algorithm to perform tokenization using the given voca

Runs basic whitespace cleaning and splitting on a piece of text.

Runs basic whitespace cleaning and splitting on a peice of text.

Runs basic whitespace cleaning and splitting on a piece of text.

Runs basic whitespace cleaning and splitting on a piece of text.

Runs basic whitespace cleaning and splitting on a piece of text.

Runs basic whitespace cleaning and splitting on a piece of text.

Runs basic whitespace cleaning and splitting on a piece of text.

Runs basic whitespace cleaning and splitting on a piece of text.

Runs basic whitespace cleaning and splitting on a piece of text.

Compute softmax probability over raw logits.

Compute softmax probability over raw logits.

Get the n-best logits from a list.

Get the n-best logits from a list.

Get the n-best logits from a list.

Get the n-best logits from a list.

Get the n-best logits from a list.

Get the n-best logits from a list.

Get the n-best logits from a list.

Sample a permutation of the factorization order, and create an attention mask accordingly. Args: inputs: int64 Tensor in shape [seq_len],

:param example: :param unique_id: :return: input_ids = (C, seq_len) token_type_ids = (C, seq_len) = s