Log `logs` on the various objects watching training, including stored metrics. Args: logs (`dict[str, float]`):

Context manager for handling null reference model (that is, peft adapter manipulation).

Context manager for handling null reference model (that is, peft adapter manipulation).

A helper function to create random tensors on the desired `device` with the desired `dtype`. When passing a list of generators, you can seed each

Predict the sample at the previous timestep by reversing the SDE. Core function to propagate the diffusion process from the learned model ou

Sets the progress bar config for the pipeline

Creates a model from a path or model identifier.

Sample instances from dataset and get prompt embeddings. Used for density ratio classifier training.

Prepare the gradienting checkpointing for the model.

Memory-efficient post-order traversal of FSDP modules to extract full parameters and sync with vLLM.

Replaces processing_class.pad_token_id to embedding_tokenizer.pad_token_id and applies self.embedding_func

r""" Apply a chat template to a conversational example along with the schema for a list of functions in `tools`. For more details, see [`mayb

Llama tokenizer does satisfy `enc(a + b) = enc(a) + enc(b)`. It does ensure `enc(a + b) = enc(a) + enc(a + b)[len(enc(a)):]`. Referen

Llama tokenizer does satisfy `enc(a + b) = enc(a) + enc(b)`. It does ensure `enc(a + b) = enc(a) + enc(a + b)[len(enc(a)):]`. Referen

Clones a chat template from a source tokenizer to the target tokenizer and updates the model accordingly. This function: - Copies the ch

Mark the history as completed.

Computes log probabilities of the reference model for a single padded batch of a KTO specific dataset.

Computes log probabilities of the reference model for a single padded batch of a BCO specific dataset.

Concatenate the `chosen` and `rejected` inputs from the batch into a single tensor for both the prompt and completion sequences.

r""" return a callable function that takes in a "messages" dataset and returns a formatted dataset, based on the tokenizer apply chat template

r""" Converts a state dict to new diffusers format. The state dict can be from previous diffusers format (`OLD_DIFFUSERS`), or PEFT format (`P

Creates a draft of a model card using the information available to the `Trainer`. Args: model_name (`str` or `None`, *op

Creates a draft of a model card using the information available to the `Trainer`. Args: model_name (`str` or `None`, *op

Decodes the input tensor and strips the padding tokens. Args: inputs (`torch.Tensor`): The input tensor to be decoded.

Flatten dictionary and concatenate nested keys with separator.

Shift non-zero elements in the mask and corresponding tensors to the right. See `flush_left` for details.

Generate responses for a list of histories.

Compute the log probabilities of the given labels under the given logits. Args: logits: Logits of the model (unnormalized). Shape

Compute the KTO loss and other metrics for the given batch of inputs for train or test.

Compute the ORPO loss and other metrics for the given batch of inputs for train or test.

Compute the BCO loss and other metrics for the given batch of inputs for train or test.

Compute the DPO loss and other metrics for the given batch of inputs for train or test.

Compute the CPO loss and other metrics for the given batch of inputs for train or test.

Load the stack-exchange-paired dataset from Hugging Face and convert it to the necessary format. The dataset is converted to a dictionary with th

Returns the trainable parameters of the pipeline

Initializes the weight update group in a distributed setup for model synchronization. Args: device (`torch.device`, `str

Light wrapper around accelerate's register_lad_state_pre_hook which is run before loading state

Whiten values with masked values.

r""" Unpair a preference dataset if it is paired. Args: dataset (`Dataset` or `DatasetDict`): Preference dataset to unpai

pre-tokenize the dataset before training; only collate during training

pre-tokenize the dataset before training; only collate during training

pre-tokenize the dataset before training; only collate during training

pre-tokenize the dataset before training; only collate during training

Resets the prefix cache for the model.

Light wrapper around accelerate's register_save_state_pre_hook which is run before saving state

Saves all of the model weights

Print the colour legend.

Split a list of indices into `chunks` chunks of roughly equal lengths.

r""" Returns the state dictionary of the model. We add the state dictionary of the value head to the state dictionary of the wrapped m

Check if the current generation sequence has finished.