move the batch to device Args: device (torch.device, str): torch device Returns: DataProto: the current Data

Pop a subset of the DataProto via `batch_keys` and `meta_info_keys` Args: batch_keys (list, optional): a list of strings indicati

Update kl_coef according to current KL.

r""" Save the pretrained model to a directory. This method is a wrapper around `transformers.PreTrainedModel.save_pretrained`. Please

r""" Return the state_dict of the pretrained model.

Compute mean of tensor with a masked values.

Split the batch among dim=0 into chunks. The meta_info is passed to each DataProto after split. Args: chunks (int): the number of

Returns the tokenizer used for tokenizing text inputs

Repeat the batch data a specified number of times. Args: repeat_times (int): Number of times to repeat the data.

Updates running moments from batch's moments computed across ranks

Report the current GPU VRAM usage.

r""" Instantiates a new model from a pretrained model from `transformers`. The pretrained model is loaded using the `from_pretrained`

Get ulysses sequence parallel process group.

Generate response with the model given the query tensor. call the `generate` method of the model. Args: query_te

This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he

Create a DataProto from a dict of tensors. This assumes that 1. All the tensor in tensors have the same dim0 2. Only dim0 is the batch

Log `logs` on the various objects watching training, including stored metrics. Args: logs (`Dict[str, float]`):

Performs a single optimization step. Args: closure (callable, optional): A closure that reevaluates the model and return

Union with another DataProto. Union batch and meta_info separately. Throw an error if - there are conflict keys in batch and they are

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

Concat a list of DataProto. The batch is concatenated among dim=0. The meta_info is assumed to be identical and will use the first one.

Run the given model on the given batch of inputs, concatenating the chosen and rejected inputs together. We do this to avoid doing two forwar

Select a subset of the DataProto via batch_keys and meta_info_keys Args: batch_keys (list, optional): a list of strings indicatin

Append dict to a dict of list.

Flatten dictionary and concatenate nested keys with separator.