Creates mismatched pairs of prompts and completions for the KL dataset by adding a +1 offset to the order of completions. For best results, t

Pack sequences in a pyarrow Table using Best Fit Decreasing strategy.

Pack sequences in a pyarrow Table using a wrapped strategy.

Process tokens of a KTO specific dataset. At this stage, we don't convert to PyTorch tensors yet; we just handle the truncation in case the promp

Process tokens of a BCO specific dataset. At this stage, we don't convert to PyTorch tensors yet; we just handle the truncation in case the promp

Tokenize a batch from a KTO specific dataset.

Tokenize a batch from a BCO specific dataset.

Tokenize a batch from a reward modelling dataset.

Reward function that checks if the completion matches the ground truth. - If both gold and prediction are parseable → use math verification.

r""" Returns: `dict` of attention processors: A dictionary containing all attention processors used in the model with

Closes the weight update group and cleans up associated resources.

Closes the communicator when weight synchronization is no longer needed. This method deletes the NCCL communicator to release associ

Closes the weight update group and cleans up the communication group.

Collate batch of examples for training.

Compute training loss and additionally compute token accuracies

r""" Computes the reward score for a given input. The method has first to enable the adapter and then compute the reward score. After

Creates a draft of a model card using the information available to the `Trainer`. Args: model_name (`str` or `None`, *op

Creates a draft of a model card using the information available to the `Trainer`. Args: model_name (`str` or `None`, *op

Creates a draft of a model card using the information available to the `Trainer`. Args: model_name (`str` or `None`, *op

Creates a draft of a model card using the information available to the `Trainer`. Args: model_name (`str` or `None`, *op

Setup the optimizer. We provide a reasonable default that works well. If you want to use something else, you can pass a tuple in the