Decodes a record to a TensorFlow example.

Whether to do layer-wise learning rate adaptation for `param_name`.

Whether to use L2 weight decay for `param_name`.

Whether to use L2 weight decay for `param_name`.

Whether to use L2 weight decay for `param_name`.

Whether to use L2 weight decay for `param_name`.

Whether to use L2 weight decay for `param_name`.

Get the variable name from the tensor name.

Get the variable name from the tensor name.

Get the variable name from the tensor name.

Get the variable name from the tensor name.

Get the variable name from the tensor name.

Checks whether CP is the codepoint of a CJK character.

Checks whether CP is the codepoint of a CJK character.

Checks whether `chars` is a control character.

Checks whether `chars` is a control character.

Checks whether `chars` is a punctuation character.

Checks whether `chars` is a punctuation character.

Check if the current word piece is the starting piece (BERT).

Check if the current word piece is the starting piece (sentence piece).

Checks whether `chars` is a whitespace character.

Checks whether `chars` is a whitespace character.

Splits punctuation on a piece of text.

Splits punctuation on a piece of text.

Strips accents from a piece of text.

Strips accents from a piece of text.

Adds whitespace around any CJK character.

Adds whitespace around any CJK character.

Truncates a sequence pair in place to the maximum length.

Truncates a sequence pair in place to the maximum length.

Truncates a sequence pair in place to the maximum length.

See base class.

See base class.

See base class.

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python in

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python intege

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python i

Raises an exception if the tensor rank is not of the expected rank. Args: tensor: A tf.Tensor to check the rank of. expected_rank: Python in

A network with attention-ffn as sub-block. Args: layer_input: float Tensor of shape [batch_size, from_seq_length, from_width]. hidden_

A network with attention-ffn as sub-block. Args: layer_input: float Tensor of shape [batch_size, from_seq_length, from_width]. hidden_

Performs multi-headed attention from `from_tensor` to `to_tensor`. Args: from_tensor: float Tensor of shape [batch_size, from_seq_length,

Performs multi-headed attention from `from_tensor` to `to_tensor`. Args: from_tensor: float Tensor of shape [batch_size, from_seq_length,

Converts a single `InputExample` into a single `InputFeatures`.

Convert a set of `InputExample`s to a list of `InputFeatures`.

Create 3D attention mask from a 2D tensor mask. Args: from_tensor: 2D or 3D Tensor of shape [batch_size, from_seq_length, ...]. to_mask: in

Creates `TrainingInstance`s for a single document.

Creates `TrainingInstance`s for a single document.

Creates `TrainingInstance`s for a single document. This method is changed to create sentence-order prediction (SOP) followed by idea from paper o

Creates the predictions for the masked LM objective.

Creates a classification model.

Creates a classification model.

Create `TrainingInstance`s from raw text.

Create `TrainingInstance`s from raw text.

Create `TrainingInstance`s from raw text.

A dense layer with 3D kernel for projection. Args: input_tensor: float Tensor of shape [batch,from_seq_length, num_attention_heads, size_p

A dense layer with 3D kernel for projection. Args: input_tensor: float Tensor of shape [batch,from_seq_length, num_attention_heads, size_p

Dot-product attention. Args: q: Tensor with shape [..., length_q, depth_k]. k: Tensor with shape [..., length_kv, depth_k]. Leading dimensio

Dot-product attention. Args: q: Tensor with shape [..., length_q, depth_k]. k: Tensor with shape [..., length_kv, depth_k]. Leading dimensio

Looks up words embeddings for id tensor. Args: input_ids: int32 Tensor of shape [batch_size, seq_length] containing word ids. vocab_si

Looks up words embeddings for id tensor. Args: input_ids: int32 Tensor of shape [batch_size, seq_length] containing word ids. vocab_si

Performs various post-processing on a word embedding tensor. Args: input_tensor: float Tensor of shape [batch_size, seq_length, embedding_

Performs various post-processing on a word embedding tensor. Args: input_tensor: float Tensor of shape [batch_size, seq_length, embedding

Performs various post-processing on a word embedding tensor. Args: input_tensor: float Tensor of shape [batch_size, seq_length, embedding_

Constructs a `AlbertConfig` from a Python dictionary of parameters.

Constructs a `BertConfig` from a Python dictionary of parameters.

Constructs a `AlbertConfig` from a Python dictionary of parameters.

Gathers the vectors at the specific positions over a minibatch.

Gathers the vectors at the specific positions over a minibatch.

Gathers the vectors at the specific positions over a minibatch.

Maps a string to a Python function, e.g., "relu" => `tf.nn.relu`. Args: activation_string: String name of the activation function. Returns:

Maps a string to a Python function, e.g., "relu" => `tf.nn.relu`. Args: activation_string: String name of the activation function. Returns:

Gets a collection of `InputExample`s for the dev set.

Gets the list of labels for this data set.

Gets the list of labels for this data set.

Get loss and log probs for the masked LM.

Get loss and log probs for the masked LM.

Get loss and log probs for the masked LM.

输入一句话，返回一句经过处理的话: 为了支持中文全称mask，将被分开的词，将上特殊标记("#")，使得后续处理模块，能够知道哪些字是属于同一个词的。 :param segment: 一句话. e.g. ['悬', '灸', '技', '术', '培', '训', '专', '家

Get loss and log probs for the next sentence prediction.

获取初步的训练实例，将整段按照max_sequence_length切分成多个部分,并以多个处理好的实例的形式返回。 :param document: 一整段 :param max_sequence_length: :return: a list. each ele

Get loss and log probs for the next sentence prediction.

Get loss and log probs for the next sentence prediction.

Gets final hidden layer of encoder. Returns: float Tensor of shape [batch_size, seq_length, hidden_size] corresponding to the final h

Gets a collection of `InputExample`s for prediction.

get total parameters of a graph :return:

Gets a collection of `InputExample`s for the train set.

Run layer normalization on the last dimension of the tensor.

Run layer normalization on the last dimension of the tensor.

Runs layer normalization followed by dropout.

Runs layer normalization followed by dropout.

Runs layer normalization followed by dropout.