Reads a tab separated value file.

Creates a `truncated_normal_initializer` with the given range.

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Converts a sequence of [tokens|ids] using the vocab.

Perform dropout. Args: input_tensor: float Tensor. dropout_prob: Python float. The probability of dropping out a value (NOT of *keepi

Creates examples for the training and dev sets.

Creates examples for the training and dev sets.

Creates examples for the training and dev sets.

Creates examples for the training and dev sets.

Converts `text` to Unicode (if it's not already), assuming utf-8 input.

Convert a set of `InputExample`s to a TFRecord file.

Creates an `input_fn` closure to be passed to TPUEstimator.

Run layer normalization on the last dimension of the tensor.

Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).

Runs basic whitespace cleaning and splitting on a piece of text.

Converts a single `InputExample` into a single `InputFeatures`.

Reshapes a rank 2 tensor back to its original rank >= 2 tensor.

Performs invalid character removal and whitespace cleanup on text.

Decodes a record to a TensorFlow example.

Whether to use L2 weight decay for `param_name`.

Get the variable name from the tensor name.

Checks whether CP is the codepoint of a CJK character.

Checks whether `chars` is a control character.

Checks whether `chars` is a punctuation character.

Checks whether `chars` is a whitespace character.

Splits punctuation on a piece of text.

Strips accents from a piece of text.

Adds whitespace around any CJK character.

Truncates a sequence pair in place to the maximum length.

Truncates a sequence pair in place to the maximum length.

Truncates a sequence pair in place to the maximum length.

Truncates a sequence pair in place to the maximum length.

See base class.

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python i

Performs multi-headed attention from `from_tensor` to `to_tensor`. This is an implementation of multi-headed attention based on "Attention is all

Converts a single `InputExample` into a single `InputFeatures`.

Converts a single `InputExample` into a single `InputFeatures`.

Converts a single `InputExample` into a single `InputFeatures`.

Create 3D attention mask from a 2D tensor mask. Args: from_tensor: 2D or 3D Tensor of shape [batch_size, from_seq_length, ...]. to_mask: in

Creates a classification model.

Looks up words embeddings for id tensor. Args: input_ids: int32 Tensor of shape [batch_size, seq_length] containing word ids. vocab_s

Performs various post-processing on a word embedding tensor. Args: input_tensor: float Tensor of shape [batch_size, seq_length, embedding

Constructs a `BertConfig` from a Python dictionary of parameters.

Constructs a `BertConfig` from a json file of parameters.

Maps a string to a Python function, e.g., "relu" => `tf.nn.relu`. Args: activation_string: String name of the activation function. Returns:

Gets a collection of `InputExample`s for the dev set.

Gets the list of labels for this data set.

Gets a collection of `InputExample`s for prediction.

Gets a collection of `InputExample`s for the train set.

Runs layer normalization followed by dropout.

Loads a vocabulary file into a dictionary.

Returns `model_fn` closure for TPUEstimator.

Serializes this instance to a Python dictionary.

Multi-headed, multi-layer Transformer from "Attention is All You Need". This is almost an exact implementation of the original Transformer encoder.

Constructs a BasicTokenizer. Args: do_lower_case: Whether to lower case the input.

Constructs a AdamWeightDecayOptimizer.

Constructs a InputExample. Args: guid: Unique id for the example. text_a: string. The untokenized text of the first sequence. For sin

Constructs a InputExample. Args: guid: Unique id for the example. text_a: string. The untokenized text of the first sequence. For sin

Constructs BertConfig. Args: vocab_size: Vocabulary size of `inputs_ids` in `BertModel`. hidden_size: Size of the encoder layers and

Constructor for BertModel. Args: config: `BertConfig` instance. is_training: bool. rue for training model, false for eval model. Cont

Constructs a InputExample. Args: guid: Unique id for the example. text_a: string. The untokenized text of the first sequence. For

Constructs a InputExample. Args: guid: Unique id for the example. text_a: string. The untokenized text of the first sequence. For sin

Convert a set of `InputExample`s to a list of `InputFeatures`.

Creates an optimizer training op.

Gaussian Error Linear Unit. This is a smoother version of the RELU. Original paper: https://arxiv.org/abs/1606.08415 Args: input_tensor: f

Compute the union of the current variables and checkpoint variables.

See base class.

See base class.

See base class.

See base class.

See base class.

Gets output of the embedding lookup (i.e., input to the transformer). Returns: float Tensor of shape [batch_size, seq_length, hidden_size]

See base class.

See base class.

See base class.

See base class.

See base class.

Gets final hidden layer of encoder. Returns: float Tensor of shape [batch_size, seq_length, hidden_size] corresponding to the final h

See base class.

See base class.