Creates a `truncated_normal_initializer` with the given range.

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

Perform dropout. Args: input_tensor: float Tensor. dropout_prob: Python float. The probability of dropping out a value (NOT of *keepi

Precision, Recall and F1 from the confusion matrix Parameters ---------- cm : tf.Tensor of type tf.int32, of shape (num_classes, num_cl

Safe division, return 0 if denominator is 0

Run layer normalization on the last dimension of the tensor.

Converts a sequence of [tokens|ids] using the vocab.

Reads a BIO data.

Converts `text` to Unicode (if it's not already), assuming utf-8 input.

Convert a set of `InputExample`s to a TFRecord file.

Creates an `input_fn` closure to be passed to TPUEstimator.

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank: (opti

Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).

Runs basic whitespace cleaning and splitting on a piece of text.

Performs multi-headed attention from `from_tensor` to `to_tensor`. This is an implementation of multi-headed attention based on "Attention is all

Creates the predictions for the masked LM objective.

Looks up words embeddings for id tensor, but in a factorized style followed by albert. it is used to reduce much percentage of parameters previous exi

Constructs a `BertConfig` from a json file of parameters.

Maps a string to a Python function, e.g., "relu" => `tf.nn.relu`. Args: activation_string: String name of the activation function. Returns:

Gets final hidden layer of encoder. Returns: float Tensor of shape [batch_size, seq_length, hidden_size] corresponding to the final h

Creates an `input_fn` closure to be passed to TPUEstimator.

Uses a confusion matrix to compute precision, recall and fbeta

Reshapes a rank 2 tensor back to its original rank >= 2 tensor.

Multi-headed, multi-layer Transformer from "Attention is All You Need". This is almost an exact implementation of the original Transformer encoder.

Truncates a pair of sequences to a maximum sequence length.

Constructs a LAMBOptimizer.

Performs invalid character removal and whitespace cleanup on text.

Decodes a record to a TensorFlow example.

Decodes a record to a TensorFlow example.

Whether to use L2 weight decay for `param_name`.

Whether to use L2 weight decay for `param_name`.

Whether to use L2 weight decay for `param_name`.

Get the variable name from the tensor name.

Get the variable name from the tensor name.

Get the variable name from the tensor name.

Checks whether CP is the codepoint of a CJK character.

Checks whether `chars` is a control character.

Checks whether `chars` is a punctuation character.

Checks whether `chars` is a whitespace character.

Splits punctuation on a piece of text.

Strips accents from a piece of text.

Adds whitespace around any CJK character.

See base class.

See base class.

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python intege

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python i

Create 3D attention mask from a 2D tensor mask. Args: from_tensor: 2D or 3D Tensor of shape [batch_size, from_seq_length, ...]. to_mask: in

Creates `TrainingInstance`s for a single document. This method is changed to create sentence-order prediction (SOP) followed by idea from paper o

Creates a classification model.

Create `TrainingInstance`s from raw text.

Performs various post-processing on a word embedding tensor. Args: input_tensor: float Tensor of shape [batch_size, seq_length, embedding

Multi-class fbeta metric for Tensorflow Parameters ---------- labels : Tensor of tf.int32 or tf.int64 The true labels pre

Constructs a `BertConfig` from a Python dictionary of parameters.

Gathers the vectors at the specific positions over a minibatch.

Get loss and log probs for the masked LM.

输入一句话，返回一句经过处理的话: 为了支持中文全称mask，将被分开的词，将上特殊标记("#")，使得后续处理模块，能够知道哪些字是属于同一个词的。 :param segment: 一句话. e.g. ['悬', '灸', '技', '术', '培', '训', '专', '家

Get loss and log probs for the next sentence prediction.

get total parameters of a graph :return:

Runs layer normalization followed by dropout.

Loads a vocabulary file into a dictionary.

Returns `model_fn` closure for TPUEstimator.

Returns `model_fn` closure for TPUEstimator.

Multi-headed, multi-layer Transformer from "Attention is All You Need". This is almost an exact implementation of the original Transformer encoder.

sentence order prediction. check method of create_instances_from_document_albert from create_pretrining_data.py :return:

Serializes this instance to a Python dictionary.

Create TF example files from `TrainingInstance`s.

Constructs a BasicTokenizer. Args: do_lower_case: Whether to lower case the input.

Constructs a AdamWeightDecayOptimizer.

Constructs a AdamWeightDecayOptimizer.

Constructs a InputExample. Args: guid: Unique id for the example. text_a: string. The untokenized text of the first sequence. For sing

Constructs BertConfig. Args: vocab_size: Vocabulary size of `inputs_ids` in `BertModel`. hidden_size: Size of the encoder layers and

Constructor for BertModel. Args: config: `BertConfig` instance. is_training: bool. true for training model, false for eval model. Con

See base class.

Convert a set of `InputExample`s to a list of `InputFeatures`.