Predict the sample at the previous timestep by reversing the SDE. Core function to propagate the diffusion process from the learned model ou

Selects the best resolution from a list of possible resolutions based on the original size. Args: original_size (tuple): The origina

Sets the progress bar config for the pipeline

Set ulysses sequence parallel process group.

Print the colour legend.

Split a list of indices into `chunks` chunks of roughly equal lengths.

r""" Returns the state dictionary of the model. We add the state dictionary of the value head to the state dictionary of the wrapped m

Perform a single step of training. Args: epoch (int): The current epoch. global_step (int): The current glob

Check if the current generation sequence has finished.

Check if the current generation sequences have finished.

Returns the text encoder used for encoding text inputs

Unpads a PyTorch tensor of a padded and resized image. Args: tensor (torch.Tensor): The image tensor, assumed to be in CxHxW format.

Check whether the text violates OpenAI moderation API.

Compute the loss. Args: model (nn.Module): The model. inputs (Dict): Batch of inputs. Returns:

Calculates the running mean and standard deviation of a data stream. Reference: https://github.com/OpenLMLab/MOSS-RLHF/blob/40b91eb2f

Add speaker and start/end signal on each round.

Add speaker and start/end signal on each round.

Reorder the data on single controller such that each dp rank gets similar total tokens

Bind the worker method to the WorkerGroup

Binds the methods of each worker to the WorkerDict. Note that we only bind public methods that are decorated by register

Check if the resource pool can be satisfied in this ray cluster.

This function should only be called inside by WorkerGroup

Patches flash attention forward to handle 3D position ids in mrope. (3, batch_size, seq_length)

r""" Handles the early stopping logic. If the policy KL is greater than the target KL, then the gradient is zeroed and the optimizatio

Calls the forward method of Flash Attention - if the input hidden states contain at least one padding token first unpad the input, th

Calls the forward method of Flash Attention - if the input hidden states contain at least one padding token first unpad the input, th

Generate responses for a list of query tensors. args: query_tensors (list[torch.Tensor]): A list of query tensors to gen

Generate samples from the model Args: iterations (int): Number of iterations to generate samples for batch_s

Compute the model size.

https://github.com/pytorch/pytorch/blob/db2a237763eb8693a20788be94f8c192e762baa8/torch/nn/functional.py#L4726

r""" Initializes the weights of the value head. The default initialization strategy is random. Users can pass a different initializati

r""" We initialize the weights of the value head.

Log a table of validation samples

Save the MeZO state checkpoint that includes the noise seeds and the accumulated grad estimates.

Check if the input data is valid for training. Args: input_ids (List[`torch.LongTensor`]): List of tenso

Check if the input data is valid for training. Args: batch_size (int): Batch size from the config file.

Train on a batch of samples. Main training segment Args: inner_epoch (int): The current inner epoch epoch (i

r""" Activates the neftune as presented in this code: https://github.com/neelsjain/NEFTune and paper: https://arxiv.org/abs/2310.05914