Split the tensor along its last dimension and keep the corresponding slice.

Zero out the gradient for a group of parameters. Note: copied from torch.optim.optimizer.

Build the model.

Return intersection-over-union (Jaccard index) of boxes. Both sets of boxes are expected to be in (x1, y1, x2, y2) format. Arguments:

Data loader. Note that batch-size is the local (per GPU) batch-size.

Buld dataloader given an input dataset.

Check for autoresume signal and exit if it is received.

Clear the embedding data structures to save memory. The metadata ends up getting used, and is also much smaller in dimensiona

1x1 convolution

1x1 convolution

1x1 convolution

3x3 convolution with padding

3x3 convolution with padding

3x3 convolution with padding

将所有候选答案放置在片段开头

Converts a sequence of ids in BPE tokens using the vocab.

Converts `text` to Unicode (if it's not already), assuming utf-8 input.

Calculate the elapsed time.

Main finetune function used across all tasks.

FP32 optimizer does not do any scaling.

Build masks and position id for left to right model.

Return tensorboard writer. It can be None so no need to check if it is initialized.

Customized load.

Returns a 2-dimensional (2-D) attention mask :param source_block: 1-D array :param target_block: 1-D array

Nanmean compatible with generators.

Nanmean compatible with generators.

Nanmean compatible with generators.

LM logits using word embedding weights.

See megatron.model.transformer.set_input_tensor()