Loads a vocabulary file into a dictionary.

Desc: marginal likelihood of gold antecedent spans form coreference cluster Args: antecedent_scores: [k, c+1

current_doc_result: "total_loss": total_loss, "start_scores": start_scores, "start_gold": gold_starts, "end_gold

Returns `model_fn` closure for TPUEstimator.

get basic information from one document annotation. :param document: :param language: english, chinese or arabic :return:

Write a InputFeature to the TFRecordWriter as a tf.train.Example.

Serializes this instance to a Python dictionary.

Tokenizes a piece of text.

tokenize into sub tokens :param doc_info: :param tokenizer: max_doc_length: pad to max_doc_length :return:

Desc: hidden_size should be equal to embeddding_size. Args: doc_overlap_window_embs: (num_window, window_siz

Multi-headed, multi-layer Transformer from "Attention is All You Need". This is almost an exact implementation of the original Transformer encode

Validate the input FLAGS or throw an exception.

Write final predictions to the json file and log-odds of null if needed.

r"""Construct a new Rectified Adam optimizer. Args: learning_rate: A Tensor or a floating point value. The learning rate.

Constructs a BasicTokenizer. Args: do_lower_case: Whether to lower case the input.

Constructs a AdamWeightDecayOptimizer.

Constructs BertConfig. Args: vocab_size: Vocabulary size of `inputs_ids` in `BertModel`. hidden_size: Size of the encoder

Constructor for BertModel. Args: config: `BertConfig` instance. is_training: bool. rue for training model, false for eval

The `model_fn` for TPUEstimator.

Creates an optimizer training op.

Gaussian Error Linear Unit. This is a smoother version of the RELU. Original paper: https://arxiv.org/abs/1606.08415 Args: input_t

Compute the union of the current variables and checkpoint variables.

Gets output of the embedding lookup (i.e., input to the transformer). Returns: float Tensor of shape [batch_size, seq_length, hidde

Desc: return the size of input x in DIM.

The actual input function.

The actual input function.

The `model_fn` for TPUEstimator.

The `model_fn` for TPUEstimator.

Returns text encoded in a way suitable for print or `tf.logging`.

Desc: transform the mention (start, end) position pairs to a span matrix gold_span_sequence_labels. matrix[i][j]