Compute the reward for a list of histories.

Concatenate the chosen and rejected inputs into a single tensor. Args: batch: A batch of data. Must contain the keys 'chosen_inpu

Converts the stats from a flattened dict to single scalar dicts

Creates and saves a model card for a TRL model. Args: path (`str`): The path to save the model card to. model_name (`

Creates and saves a model card for a TRL model. Args: path (`str`): The path to save the model card to. model_name (`

Divides an image into patches of a specified size. Args: image (PIL.Image.Image): The input image. patch_size (int): The siz

Compute the DPO loss for a batch of policy and reference model log probabilities. Args: policy_chosen_logps: Log probabilities of

Calculate entropy from logits.

Gather stats from all processes. Useful in the context of distributed training. Args: stats (dict[str, Any]):

r""" A simple wrapper around the `generate` method of the wrapped model. Please refer to the [`generate`](https://huggingface.co/docs/

Compute the log probabilities of the given labels under the given logits. Args: logits: Logits of the model (unnormalized). Shape

Generate samples from the model and reference model for the given batch of inputs.

r""" Finds the correct formatting function based on the dataset structure. Currently supported datasets are: - `ChatML` with [{"role": str, "c

Computes element-wise mean and variance of the tensor across processes. Reference: https://github.com/OpenLMLab/MOSS-RLHF/blob/40b91eb2f2b71b

Return a list of indices so that each slice of `batch_size` consecutive indices correspond to elements of similar lengths. To do this, the in

Return a list of indices so that each slice of `batch_size` consecutive indices correspond to elements of similar lengths. To do this, the in

Returns the trainable parameters of the pipeline

r""" return a callable function that takes in an "instructions" dataset and returns a formatted dataset, based on the tokenizer apply chat tem

Light wrapper around accelerate's register_lad_state_pre_hook which is run before loading state

Calculate policy and value losses. Args: old_logprobs (`torch.FloatTensor`): Log probabilities of the mo

Make dataset and collator for supervised fine-tuning.

Make dataset and collator for supervised fine-tuning.

Parse command-line arguments.

Parse command-line arguments.

Parse request string. Expected format: <request><tool_name>query<call>

r""" Function invoked when calling the pipeline for generation. Args: prompt (`str` or `List[str]`, *optional*): The prompt or prompts to guide t

Prepare the dataloader for training. Args: dataset (Union[`torch.utils.data.Dataset`, `datasets.Dataset`]):

Given a list of sources, each is a conversation list. This transform: 1. Add signal '### ' at the beginning each sentence, with end signal '\