Creates a `truncated_normal_initializer` with the given range.

Reads a tab separated value file.

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Creates examples for the training and dev sets.

Converts a sequence of [tokens|ids] using the vocab.

Perform dropout. Args: input_tensor: float Tensor. dropout_prob: Python float. The probability of dropping out a value (NOT of *keepi

Creates examples for the training and dev sets.

Converts `text` to Unicode (if it's not already), assuming utf-8 input.

Convert a set of `InputExample`s to a TFRecord file.

Creates an `input_fn` closure to be passed to TPUEstimator.

Run layer normalization on the last dimension of the tensor.

Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).

Runs basic whitespace cleaning and splitting on a piece of text.

Converts a single `InputExample` into a single `InputFeatures`.

Creates the predictions for the masked LM objective.

Constructs a `BertConfig` from a json file of parameters.

输入一句话，返回一句经过处理的话: 为了支持中文全称mask，将被分开的词，将上特殊标记("#")，使得后续处理模块，能够知道哪些字是属于同一个词的。 :param segment: 一句话 :return: 一句处理过的话

Creates an `input_fn` closure to be passed to TPUEstimator.

Reshapes a rank 2 tensor back to its original rank >= 2 tensor.

Performs invalid character removal and whitespace cleanup on text.

Decodes a record to a TensorFlow example.

Decodes a record to a TensorFlow example.

Whether to use L2 weight decay for `param_name`.

Whether to use L2 weight decay for `param_name`.

Get the variable name from the tensor name.

Get the variable name from the tensor name.

Checks whether CP is the codepoint of a CJK character.

Checks whether `chars` is a control character.

Checks whether `chars` is a punctuation character.

Checks whether `chars` is a whitespace character.

Splits punctuation on a piece of text.

Strips accents from a piece of text.

Adds whitespace around any CJK character.

Truncates a sequence pair in place to the maximum length.

See base class.

See base class.

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python i

Performs multi-headed attention from `from_tensor` to `to_tensor`. This is an implementation of multi-headed attention based on "Attention is all

Create 3D attention mask from a 2D tensor mask. Args: from_tensor: 2D or 3D Tensor of shape [batch_size, from_seq_length, ...]. to_mask: in

Creates `TrainingInstance`s for a single document.

Creates a classification model.

Create `TrainingInstance`s from raw text.

Looks up words embeddings for id tensor. Args: input_ids: int32 Tensor of shape [batch_size, seq_length] containing word ids. vocab_s

Performs various post-processing on a word embedding tensor. Args: input_tensor: float Tensor of shape [batch_size, seq_length, embedding

Constructs a `BertConfig` from a Python dictionary of parameters.

Gathers the vectors at the specific positions over a minibatch.

Maps a string to a Python function, e.g., "relu" => `tf.nn.relu`. Args: activation_string: String name of the activation function. Returns:

Gets a collection of `InputExample`s for the dev set.

Gets the list of labels for this data set.

Get loss and log probs for the masked LM.

Get loss and log probs for the next sentence prediction.

获取初步的训练实例，将整段按照max_sequence_length切分成多个部分,并以多个处理好的实例的形式返回。 :param document: 一整段 :param max_sequence_length: :return: a list. each ele

Gets final hidden layer of encoder. Returns: float Tensor of shape [batch_size, seq_length, hidden_size] corresponding to the final h

Gets a collection of `InputExample`s for prediction.

Gets a collection of `InputExample`s for the train set.

Runs layer normalization followed by dropout.

Loads a vocabulary file into a dictionary.

Returns `model_fn` closure for TPUEstimator.

Returns `model_fn` closure for TPUEstimator.

Serializes this instance to a Python dictionary.

Multi-headed, multi-layer Transformer from "Attention is All You Need". This is almost an exact implementation of the original Transformer encoder.

Truncates a pair of sequences to a maximum sequence length.

Create TF example files from `TrainingInstance`s.

Constructs a BasicTokenizer. Args: do_lower_case: Whether to lower case the input.

Constructs a InputExample. Args: guid: Unique id for the example. text_a: string. The untokenized text of the first sequence. For sing

Constructs a AdamWeightDecayOptimizer.

Constructs a AdamWeightDecayOptimizer.

Constructs BertConfig. Args: vocab_size: Vocabulary size of `inputs_ids` in `BertModel`. hidden_size: Size of the encoder layers and

Constructor for BertModel. Args: config: `BertConfig` instance. is_training: bool. true for training model, false for eval model. Con

Checks whether CP is the codepoint of a CJK character.

Convert a set of `InputExample`s to a list of `InputFeatures`.

Creates `TrainingInstance`s for a single document.

Creates an optimizer training op.

Creates an optimizer training op.

get trainable variables only to train from layer 9 to last layer :param tvars: a list :return: a new tvars, which is list

Gaussian Error Linear Unit. This is a smoother version of the RELU. Original paper: https://arxiv.org/abs/1606.08415 Args: x: float Tensor

Compute the union of the current variables and checkpoint variables.

See base class.

See base class.

Gets output of the embedding lookup (i.e., input to the transformer). Returns: float Tensor of shape [batch_size, seq_length, hidden_size]