Parse request string. Expected format: <request><tool_name>query<call>

Registers a LightEval task configuration. - Core tasks can be added from this table: https://github.com/huggingface/lighteval/blob/main/src/light

Resets the prefix cache for the model.

Reward function that checks if we produce the desired number of think and answer tags associated with `format_reward()`. Adapted from: https://gi

Generate samples from the model Args: iterations (int): Number of iterations to generate samples for batch_s

TODO: optimize this. Technically, we only need one broadcast

Runs the given model on the given batch of inputs, concatenating the chosen and rejected inputs together. We do this to avoid doing

Takes an N-dimensional bool tensor and returns an (N-1)-dimensional tensor of integers giving the position of the first True in each "row".

Shift non-zero elements in the mask and corresponding tensors to the right. See `flush_left` for details.

Performs a forward pass through the model with the given query responses and pad token ID. Args: model (`torch.nn.Module`):

Build the model.

Get ulysses sequence parallel world size.

Initializes the weight update group in a distributed setup for model synchronization.

Compute length-based rewards to discourage overthinking and promote token efficiency. Taken from the Kimi 1.5 tech report: https://huggingface.co

Split the batch into a list of micro_batches, where the max_token_len is smaller than max_token_len and the number of valid tokens in each micro b

Calculates the raw score for the subtask as the minimum score across all test results. Returns: float: The rounded minim

Step the environment forward one turn. Args: history (`TextHistory`): The history to step forward.

Truncates the input tensor from the right side after the first occurrence of the stop token. Args: input_ids (`torch.Tensor`):

Union with another DataProto. Union batch and meta_info separately. Throw an error if - there are conflict keys in batch and they are

Updates all parameters of the given model by calling `update_named_param` for each parameter in the model. Args: model (

broadcast tensor across mp_group

broadcast tensor in tp shards across mp_group

Reward function that checks if the completion is the same as the ground truth.

TODO: optimize this. - We can use async ops - We can use only one allgather Args: tensors: size: group:

Closes the weight update group and cleans up the communication group.

Mark the history as completed.

Compute logits given a batch of data. Args: data (DataProto): a batch of data represented by DataProto. It must contain key ```in

Decodes the input tensor and strips the padding tokens. Args: inputs (`torch.Tensor`): The input tensor to be decoded.

Compute the log probabilities of the given labels under the given logits. Args: logits: Logits of the model (unnormalized). Shape

Get the resource pool of the worker_cls

Reward function that penalizes overlong completions. It is used to penalize overlong completions, but not to reward shorter completions. Refe

Convert a conversational dataset with fields `from` and `value` to ChatML format. This function modifies conversational data to align with O

r""" Extracts the shared prompt from a preference data example, where the prompt is implicit within both the chosen and rejected completions.

A context manager function for profiling a block of code. Results are logged to Weights & Biases or MLflow depending on the trainer's configu

This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he

Rotates half the hidden dims of the input.

Shuffles a dictionary of tensors along the first dimension in unison. Example: >>> x = torch.arange(6).reshape(3, 2) >>> y =

Returns the text encoder used for encoding text inputs

r""" Truncate sequences in a dataset to a specifed `max_length`. Args: dataset (`Dataset` or `DatasetDict`): Dataset to t

Truncates the responses at the first occurrence of the stop token, filling the rest with pad tokens. Args: stop_token_id (`int`):

Pad and slice input_ids to be divisible by sp_size Pad position_ids to be divisible by sp_size. Note both input_ids_rmpad and position_i

Returns the 2d U-Net model used for diffusion.

Updates a specific named parameter in the model and broadcasts it to other processes. Args: name (`str`):

broadcast tensor in tp shards across mp_group

Compute KL log probabilities for a given batch.

Generate responses for a list of query tensors. Args: query_tensors (list[torch.Tensor]): A list of query tensors to gen

Generates completions for a list of pre-formatted prompts from the given model. Args: prompts (list[str]): A list of input prompts f

r""" Get the current device. For GPU, we return the local process index using the `accelerate.PartialState` object to handle corner ca

As opposed to the default direction of broadcasting (right to left), this function broadcasts from left to right Args: in