Make dataset and collator for supervised fine-tuning.

Parse the prediction from the generated response. Return a list of predicted strings or numbers.

Given a list of sources, each is a conversation list. This transform: 1. Add signal '### ' at the beginning each sentence, with end signal '\

Calculates relaxed correctness. The correctness tolerates certain error ratio defined by max_relative_change. See https://arxiv.org/pdf/2203.

Collects the state dict and dump to disk.

Split scope and key. The first scope will be split from key. Examples: >>> Registry.split_scope_key('mmdet.ResNet')

Split scope and key. The first scope will be split from key. Examples: >>> Registry.split_scope_key('mmdet.ResNet')

Split a list of indices into `chunks` chunks of roughly equal lengths.

Converts 2D attention mask to 4D attention mask by expanding mask to (bsz, head_dim=1, query_length, key_value_length) shape and by a

Tokenize prompts and make them avaiable on all ranks.

Tokenize prompts and make them avaiable on all ranks.

Build train and valid datasets.