r""" Instantiates a new model from a pretrained model from `transformers`. The pretrained model is loaded using the `from_pretrained`

Updates running moments from batch's moments computed across ranks

r""" Save the pretrained model to a directory. This method is a wrapper around `transformers.PreTrainedModel.save_pretrained`. Please

r""" Return the state_dict of the pretrained model.

Compute mean of tensor with a masked values.

Returns the tokenizer used for tokenizing text inputs

Log `logs` on the various objects watching training, including stored metrics. Args: logs (`Dict[str, float]`):

r""" Post initialization method. This method is called after the model is instantiated and loaded from a checkpoint. It can be used to

Run the given model on the given batch of inputs, concatenating the chosen and rejected inputs together. We do this to avoid doing two forwar

Return a list of indices so that each slice of `batch_size` consecutive indices correspond to elements of similar lengths. To do this, the in

Flatten dictionary and concatenate nested keys with separator.

Generate response with the model given the query tensor. call the `generate` method of the model. Args: query_te

Checks if `torch_npu` is installed and potentially if a NPU is in the environment

r""" Get the current device. For GPU, we return the local process index using the `accelerate.PartialState` object to handle corner ca

As opposed to the default direction of broadcasting (right to left), this function broadcasts from left to right Args: in

Separate the kwargs from the arguments that we support inside `supported_args` and the ones that we don't.

Append a new segment to the history. args: text (`str`): The text of the new segment. tokens (`torch.LongTen

Calculate model outputs in multiple batches. Args: queries (`torch.LongTensor`): List of tensors contain

See: https://github.com/pytorch/pytorch/issues/563#issuecomment-330103591

Compute variance of tensor with masked values.

Context manager for handling null reference model (that is, peft adapter manipulation).

Process an image with variable resolutions. Args: image (PIL.Image.Image): The input image to be processed. processor: The i

Returns the 2d U-Net model used for diffusion.

Tokenize a list of strings.

Tokenize a list of strings.

Returns the autocast context manager

Llama tokenizer does satisfy `enc(a + b) = enc(a) + enc(b)`. It does ensure `enc(a + b) = enc(a) + enc(a + b)[len(enc(a)):]`.

Mark the history as completed.

Computes log probabilities of the reference model for a single padded batch of a DPO specific dataset.

Compute per token rewards from scores and KL-penalty. Args: scores (`torch.FloatTensor`): Scores from th

r""" return a callable function that takes in a "messages" dataset and returns a formatted dataset, based on the tokenizer apply chat template

Creates a static reference copy of a model. Note that model will be in `.eval()` mode. Args: model (`PreTrainedModelWrapper`): The m