Check if the current word piece is the starting piece (BERT).

selectively load retrieval models for indexing/retrieving from saved checkpoints

Populate members from instance saved to file

Returns a 2-dimensional (2-D) attention mask :param source_block: 1-D array :param target_block: 1-D array

Returns a 2-dimensional (2-D) attention mask :param source_block: 1-D array :param target_block: 1-D array

Map the validation dataset Table like name of file -> it's id in the W&B Table. Useful for - referencing artifacts for evaluation.

Returns mixed inputs, pairs of targets, and lambda

Returns mixed inputs, pairs of targets, and lambda

pre_pred:已经得到的前一个预测结果 这里要将原始已经得到的预测结果逐个加入原始测试集

Returns text encoded in a way suitable for print or `tf.logging`.

Process batch and produce inputs for the model.

Process batch and produce inputs for the model.

将原始数据处理为如下形式： part_passage遍历每个blak的周围位置 :param input_data: :param is_training: :return:

Receive tensor from previous rank in pipeline (forward receive).

:param data: [N, 12, 24, 72]

:param data: [N, 12, 24, 72]

:param data: [N, 12, 24, 72]

Replace BatchNorm modules in input model and copy BatchNorm parameters. Current implementation only supports normal BN to FrozenBN :param

Reset timer.

r"""ResNet-50 model from `"Deep Residual Learning for Image Recognition" <https://arxiv.org/pdf/1512.03385.pdf>`_ Args: pretrained (b

Send tensor to next rank in pipeline (forward send).

Set args, tokenizer, tensorboard-writer, adlr-autoresume, and timers.

Return a view of the list of tokens from [i, j).

Save dict with state dicts of each of the models.

For easy load when model is combined with other heads, add an extra key.

Save dict with state dicts of each of the models.

Tempered softmax function. Args: activations: A multi-dimensional tensor with last dimension `num_classes`. t: Temperature > 1.0.