Loads a vocabulary file into a dictionary.

Loads a vocabulary file into a dictionary.

Returns `model_fn` closure for TPUEstimator.

Returns `model_fn` closure for TPUEstimator.

Returns `model_fn` closure for TPUEstimator.

Returns `model_fn` closure for TPUEstimator.

Returns `model_fn` closure for TPUEstimator.

Returns `model_fn` closure for TPUEstimator.

Returns `model_fn` closurimport_tfe for TPUEstimator.

Multi-headed, multi-layer Transformer from "Attention is All You Need". This is almost an exact implementation of the original Transformer encoder.

sentence order prediction. check method of create_instances_from_document_albert from create_pretrining_data.py :return:

Serializes this instance to a Python dictionary.

Serializes this instance to a Python dictionary.

Serializes this instance to a Python dictionary.

Multi-headed, multi-layer Transformer from "Attention is All You Need". This is almost an exact implementation of the original Transformer encoder.

Multi-headed, multi-layer Transformer from "Attention is All You Need". This is almost an exact implementation of the original Transformer encoder.

Truncates a pair of sequences to a maximum sequence length.

Truncates a pair of sequences to a maximum sequence length.

Create TF example files from `TrainingInstance`s.

Create TF example files from `TrainingInstance`s.

Create TF example files from `TrainingInstance`s.

Constructs a LAMBOptimizer.

Constructs a AdamWeightDecayOptimizer.

Constructs AlbertConfig. Args: vocab_size: Vocabulary size of `inputs_ids` in `AlbertModel`. embedding_size: size of voc embeddings.

Constructor for AlbertModel. Args: config: `AlbertConfig` instance. is_training: bool. true for training model, false for eval model.

Constructs a BasicTokenizer. Args: do_lower_case: Whether to lower case the input.

Constructs a InputExample. Args: guid: Unique id for the example. text_a: string. The untokenized text of the first sequence. For sing

Constructs a InputExample. Args: guid: Unique id for the example. text_a: string. The untokenized text of the first sequence. For sing

Constructs a BasicTokenizer. Args: do_lower_case: Whether to lower case the input.

Constructs a AdamWeightDecayOptimizer.

Constructs a AdamWeightDecayOptimizer.

Constructs BertConfig. Args: vocab_size: Vocabulary size of `inputs_ids` in `BertModel`. hidden_size: Size of the encoder layers and

Constructor for BertModel. Args: config: `BertConfig` instance. is_training: bool. true for training model, false for eval model. Con

Constructs a InputExample. Args: guid: Unique id for the example. text_a: string. The untokenized text of the first sequence. For sing

Constructs AlbertConfig. Args: vocab_size: Vocabulary size of `inputs_ids` in `AlbertModel`. embedding_size: size of voc embeddings.

Constructor for AlbertModel. Args: config: `AlbertConfig` instance. is_training: bool. true for training model, false for eval model.

Checks whether CP is the codepoint of a CJK character.

Reads a tab separated value file.

Reads a tab separated value file.

See base class.

See base class.

Convert a set of `InputExample`s to a list of `InputFeatures`.

Convert a set of `InputExample`s to a list of `InputFeatures`.

Converts a single `InputExample` into a single `InputFeatures`.

Creates `TrainingInstance`s for a single document.

Creates `TrainingInstance`s for a single document.

Creates the predictions for the masked LM objective.

Creates an optimizer training op.

Creates an optimizer training op.

Creates an optimizer training op.

A dense layer with 2D kernel. 添加分组全连接的方式 Args: input_tensor: Float tensor with rank 3. [ batch_size,sequence_length, hidden_size] output_siz

Looks up words embeddings for id tensor. Args: input_ids: int32 Tensor of shape [batch_size, seq_length] containing word ids. vocab_s

Constructs a `AlbertConfig` from a json file of parameters.

Constructs a `AlbertConfig` from a json file of parameters.

Gathers the vectors at the specific positions over a minibatch.

Gaussian Error Linear Unit. This is a smoother version of the RELU. Original paper: https://arxiv.org/abs/1606.08415 Args: x: float Tensor t

Gaussian Error Linear Unit. This is a smoother version of the RELU. Original paper: https://arxiv.org/abs/1606.08415 Args: x: float Tensor

Gaussian Error Linear Unit. This is a smoother version of the RELU. Original paper: https://arxiv.org/abs/1606.08415 Args: x: float Tensor t

Compute the union of the current variables and checkpoint variables.

Compute the union of the current variables and checkpoint variables.

Compute the union of the current variables and checkpoint variables.

Gets a collection of `InputExample`s for the dev set.

See base class.

See base class.

See base class.