Generate responses for a list of histories.

Calculate the shape of the image patch grid after the preprocessing for images of any resolution. Args: image_size (tuple): The size

Compute the DPO loss and other metrics for the given batch of inputs for train or test. CHANGE: 1. add sft loss 2. all gather metrics

Makes broadcastable attention and causal masks so that future and masked tokens are ignored. Arguments: attention_mask (

Whiten values with masked values.

Given a list of sources, each is a conversation list. This transform: 1. Add signal '### ' at the beginning each sentence, with end signal '\

Perform per-sample random masking by per-sample shuffling. Per-sample shuffling is done by argsort random noise. x: [N, L, D]

Run the environment on a list of queries. Args: queries (list[str]): A list of queries to run the model in the environme

Saves all of the model weights

Predict the sample at the previous timestep by reversing the SDE. Core function to propagate the diffusion process from the learned model ou

Selects the best resolution from a list of possible resolutions based on the original size. Args: original_size (tuple): The origina

Sets the progress bar config for the pipeline

Helper function for reproducible behavior to set the seed in `random`, `numpy`, and `torch`. Args: seed (`int`): The seed to set.

Print the colour legend.

Split a list into n (roughly) equal-sized chunks

Split a list into n (roughly) equal-sized chunks

Split a list of indices into `chunks` chunks of roughly equal lengths.

r""" Returns the state dictionary of the model. We add the state dictionary of the value head to the state dictionary of the wrapped m

Perform a single step of training. Args: epoch (int): The current epoch. global_step (int): The current glob

Check if the current generation sequence has finished.

Check if the current generation sequences have finished.

Returns the text encoder used for encoding text inputs

Calculates the running mean and standard deviation of a data stream. Reference: https://github.com/OpenLMLab/MOSS-RLHF/blob/40b91eb2f

Add speaker and start/end signal on each round.

Add speaker and start/end signal on each round.

r""" Handles the early stopping logic. If the policy KL is greater than the target KL, then the gradient is zeroed and the optimizatio

Generate responses for a list of query tensors. args: query_tensors (list[torch.Tensor]): A list of query tensors to gen

Generate samples from the model Args: iterations (int): Number of iterations to generate samples for batch_s

r""" Initializes the weights of the value head. The default initialization strategy is random. Users can pass a different initializati

r""" We initialize the weights of the value head.

Check if the input data is valid for training. Args: input_ids (List[`torch.LongTensor`]): List of tenso

Check if the input data is valid for training. Args: batch_size (int): Batch size from the config file.

Train on a batch of samples. Main training segment Args: inner_epoch (int): The current inner epoch epoch (i

r""" Activates the neftune as presented in this code: https://github.com/neelsjain/NEFTune and paper: https://arxiv.org/abs/2310.05914

r""" Add and load a reward modeling adapter. This method can only be used if the model is a `PeftModel` and if you have initialized th

Calculate the loss for a batch of an unpacked sample Args: latents (torch.Tensor): The latents sampled f

Tensor extension to torch.clamp https://github.com/pytorch/pytorch/issues/2793#issuecomment-428784713