Flatten dictionary and concatenate nested keys with separator.

Args: input_ids: input ids. shape (batch_size, seq_length) attention_mask: attention_mask. shape (batch_size, seq_le

We assume: - The model takes input: (input_ids, attention_mask, position_ids). No rmpad for the input - The communication sha

A func to sync attention result with alltoall in sequence parallel gather head dimension and scatter seq dim: e.g. seq_dim: 1, head_dim:

Generate responses for a list of histories.

r""" Generate the best of n samples for input queries Args: tokenized_query (`list[int]` or `torch.Tensor` or `list[torch

Get all the parameters of the models in all pp ranks Returns: params: List[List[Dict[str, Tensor]]]: a list of parameters in all

Compute the log probabilities of the given labels under the given logits. Args: logits: Logits of the model (unnormalized). Shape

Compute the KTO loss and other metrics for the given batch of inputs for train or test.

Compute the ORPO loss and other metrics for the given batch of inputs for train or test.

Compute the BCO loss and other metrics for the given batch of inputs for train or test.

Compute the DPO loss and other metrics for the given batch of inputs for train or test.

Compute the CPO loss and other metrics for the given batch of inputs for train or test.

e.g. end of sentence token=1 response_id: [0, 0, 2, 42, 3, 5, 1, 0, 0] eos_mask: [1, 1, 1, 1, 1, 1, 1, 0, 0]

Get the model

get order of seq lengths to make partitions balanced, this is used in balacing sum of seqlength across dp ranks and microbatches Paramete

Load the stack-exchange-paired dataset from Hugging Face and convert it to the necessary format. The dataset is converted to a dictionary with th

Returns the trainable parameters of the pipeline

Return a dictionary containing name to a shape and dtype.

Helper function for setting up Weights & Biases logging tools.

Light wrapper around accelerate's register_lad_state_pre_hook which is run before loading state

load all model params to cuda

Make an iterator from the DataProto. This is built upon that TensorDict can be used as a normal Pytorch dataset. See https://pytorch.org/tenso

Make minibatch iterator for updating the actor Args: data (DataProto): a DataProto containing keys ``input_ids``

r"""Works like os.makedirs() but supports hdfs. Super-mkdir; create a leaf directory and all intermediate ones. Works like mkdir, except tha

Compute variance of tensor with masked values.

Compute a similarity score considering element frequency, ignoring order.

Normalize the pp vpp params into a complete named parameters. This is useful when gather parameters from pp ranks and passed to a model with

Basic string normalization.

Context manager for handling null reference model (that is, peft adapter manipulation).

Pack examples into chunks of size `seq_length`. Args: examples (`dict[str, list[list]]`): Dictionary of examples with ke

pre-tokenize the dataset before training; only collate during training

pre-tokenize the dataset before training; only collate during training

pre-tokenize the dataset before training; only collate during training

pre-tokenize the dataset before training; only collate during training

Execute multiple scripts using MorphCloud via the router. Args: scripts: List of code scripts to execute. la

Light wrapper around accelerate's register_save_state_pre_hook which is run before saving state

Saves all of the model weights

Scores all test cases in a subtask. Args: client: PistonClient instance for executing code subtask: Dictionary containing su

Overrides the parser's default values with those provided via keyword arguments, including for subparsers. Any argument with an upda

Set ulysses sequence parallel process group.

Print the colour legend.

Split the tokens into query and response tokens.

r""" Returns the state dictionary of the model. We add the state dictionary of the value head to the state dictionary of the wrapped m

Check if the current generation sequence has finished.

Check if the current generation sequences have finished.

Converts the SubtaskResult to a dictionary representation. Returns: dict: Dictionary containing all subtask result data

Update the policy with an iterator of DataProto Args: data (DataProto): an iterator over the DataProto that returns by

Generate word-level n-grams from text.