Creates a random int32 tensor of the shape within the vocab size.

Example: >>> init_logger(log_file) >>> logger.info("abc'")

Run layer normalization on the last dimension of the tensor.

Load sentences. A line must contain at least a word and its tag. Sentences are separated by empty lines.

Plots the learning rate range test. Arguments: skip_start (int, optional): number of batches to trim from the start.

Post-attention processing.

Prepare the dataset. Return a list of lists of dictionaries containing: - word indexes - word char indexes - tag indexes

计算混淆矩阵

将结果用表格的形式打印出来，像这个样子： precision recall f1-score support avg/total

Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).

:param sess: session to run the batch :param is_train: a flag indicate if it is a train batch :param batch: a dict containing

Performs a single optimization step. Arguments: closure (callable, optional): A closure that reevaluates the model

Make sure we are sharing the input and output embeddings. Export to TorchScript can't handle parameter sharing so we are cloning them

Serializes this instance to a JSON string.

Tokenizes a piece of text into its word pieces. This uses a greedy longest-match-first algorithm to perform tokenization using the given voca

Runs basic whitespace cleaning and splitting on a piece of text.

Runs basic whitespace cleaning and splitting on a piece of text.

Runs basic whitespace cleaning and splitting on a piece of text.

Get the n-best logits from a list.

Resets wait counter and cooldown counter.

Converts a string in a sequence of tokens (string), using the tokenizer. Split in words for word-based vocabulary or sub-words for sub-wo

Add a list of new tokens to the tokenizer class. If the new tokens are not in the vocabulary, they are added to it with indices start

Build model inputs from a sequence or a pair of sequence for sequence classification tasks by concatenating and adding special tokens

Loads a data file into a list of `InputBatch`s.

Loads a data file into a list of `InputBatch`s `cls_token_at_end` define the location of the CLS token: - False (Default, BERT/XL

Converts a single index or a sequence of indices (integers) in a token " (resp.) a sequence of tokens (str/unicode), using the vocabulary

将一个样本进行分析，然后将字转化为id, 标签转化为lb

Converts a single `InputExample` into a single `InputFeatures`.

Converts a sequence of tokens (string) in a single string. The most simple way to do it is ' '.join(self.convert_ids_to_tokens(token_ids)

Create a dictionary of items from a list of list of items.

Find the most likely tag sequence using Viterbi algorithm. Args: emissions (`~torch.Tensor`): Emission score tensor of size

turn sentences into word pieces.

Gets a collection of `InputExample`s for the dev set.

Gets the list of labels for this data set.

Return set of symbol pairs in a word. word is represented as tuple of symbols (symbols being variable-length strings)

Return set of symbol pairs in a word. Word is represented as tuple of symbols (symbols being variable-length strings).

Return set of symbol pairs in a word. Word is represented as tuple of symbols (symbols being variable-length strings).

Return set of symbol pairs in a word. word is represented as tuple of symbols (symbols being variable-length strings)

Gets a collection of `InputExample`s for prediction.

Gets a collection of `InputExample`s for the train set.

Creates an `input_fn` closure to be passed to TPUEstimator.

Creates an `input_fn` closure to be passed to TPUEstimator.

将json list写入text文件中 :param file_path: :param data: :return:

Make folders for training and evaluation

Position-wise Feed-forward Network.

Returns text encoded in a way suitable for print or `tf.logging`.