Check if the current generation sequences have finished.

r""" Tokenize a row of the dataset. Args: features (`dict[str, str]`): Row of the dataset, should contain

Filter a distribution of logits using top-k and/or nucleus (top-p) filtering Args: logits: logits distribution shape (batch size, vocabula

Compute the KTO loss using the Liger-Kernel's LigerFusedLinearKTOLoss. Args: model: The policy model use

Creates a model from a path or model identifier.

Creates a model from a path or model identifier.

Enables gradient checkpointing for the model.

Enables gradient checkpointing for the model.

Generate responses for a list of query tensors. Args: query_tensors (list[torch.Tensor]): A list of query tensors to gen

Generate samples from the model Args: iterations (int): Number of iterations to generate samples for batch_s

Generate samples from the model Args: batch_size (int): Batch size to use for sampling with_grad (bool): Whe

Calculates the probability if the given prompt embedding is from desirable dataset. This function calculates the probability in the p

Extract embeddings from frozen embedding model

r""" Initializes the weights of the value head. The default initialization strategy is random. Users can pass a different initializati

r""" We initialize the weights of the value head.

Interpolate mu, sigma and ratio given training progress in [0,1].

Prepares a quantized model for kbit training.

Prepares a model for PEFT training.

Prepares a model for PEFT training.

Check if the input data is valid for training. Args: input_ids (list[`torch.LongTensor`]): List of tenso

Train on a batch of samples. Main training segment Args: inner_epoch (int): The current inner epoch epoch (i

r""" Add and load a reward modeling adapter. This method can only be used if the model is a `PeftModel` and if you have initialized th

Adds the optimizer hooks from a DeepSpeed ZeRO-3 model.

Compute the BCO loss for a batch of policy and reference model log probabilities. Args: policy_chosen_logps: Log

Build dataset for training. This builds the dataset from `load_dataset`, one should customize this function to train the model on its own dat

Build dataset for training. This builds the dataset from `load_dataset`, one should customize this function to train the model on its own dat

Calculate the loss for a batch of an unpacked sample Args: latents (torch.Tensor): The latents sampled f

Calculate the loss for a batch of an unpacked sample Args: rewards (torch.Tensor): Differentiable reward

Estimate the average number of characters per token in the dataset.

Estimate the average number of characters per token in the dataset.

Check server availability with retries on failure, within a total timeout duration. If the server is not up after the total timeout d

Split list `lst` into `n` evenly distributed sublists. Example: ```python >>> chunk_list([1, 2, 3, 4, 5, 6], 2) [[1, 2, 3], [4,

Compute the reward for a list of histories.

Run the given model on the given batch of inputs, concatenating the chosen and rejected inputs together. We do this to avoid doing two forwar

Run the given model on the given batch of inputs, concatenating the chosen and rejected inputs together. We do this to avoid doing two forwar

Concatenate the chosen and rejected inputs into a single tensor. Args: batch: A batch of data. Must contain the k

Concatenate the chosen and rejected inputs into a single tensor. Args: batch: A batch of data. Must contain the k

r""" Simply iterates over the state dict and replaces the patterns in `mapping` with the corresponding values. Args: state_dict (`dic

Compute the CPO loss for a batch of policy and reference model log probabilities. Args: policy_chosen_logps: Log

Creates a merge configuration for a DARE TIES merge of two models, with specified weights and densities.