Computes throughput metrics for PPO training. This function calculates performance metrics related to token processing speed, includ

Computes timing metrics for different processing stages in PPO training. This function calculates both raw timing metrics (in seconds) a

Convert Hydra configs and other special types to regular Python types.

Convert Hydra configs and other special types to regular Python types.

Create a sampler for the dataset. Arguments: data_config: The data config. dataset (Dataset): The dataset. Returns:

Create a sampler for the dataset. Arguments: data_config: The data config. dataset (Dataset): The dataset. Returns:

Enable activation offloading for the model. It groups activations by TransformerLayer and offloads activation groups asynchronously. This mea

Execute a method on all workers asynchronously. Args: method_name: Name of the method to execute *args: Positional ar

Execute a method on all workers synchronously. Args: method_name: Name of the method to execute *args: Positional arg

Return the most recent checkpoint directory based on a tracker file. Args: path (str): Base directory containing the checkpoint trac

Return the most recent checkpoint directory based on a tracker file. Args: path (str): Base directory containing the checkpoint trac

The training loop of PPO. The driver process only need to call the compute functions of the worker group through RPC to const

The training loop of PPO. The driver process only need to call the compute functions of the worker group through RPC to const

Synchronous implementation of the tool functionality. Override this method for synchronous tools. Returns:

Synchronous implementation of the tool functionality. Override this method for synchronous tools. Returns:

Create a worker group from existing detached workers. Args: name_prefix: Prefix for worker names worker_names: Names

torch.nn.utils.clip_grad_norm_ cann't run on cpu parameter DTensor

torch.nn.utils.clip_grad_norm_ cann't run on cpu parameter DTensor

Create a constant LR schedule with a linear warmup phase. Args: optimizer (Optimizer): Wrapped optimizer. num_warmup_steps (

Create a schedule with a learning rate that decreases following the values of the cosine function between the initial lr set in the optimizer

Get the forward function for given model architecture.

Get the forward function for given model architecture.

Calculates partitions of indices from seqlen_list such that the sum of sequence lengths in each partition is balanced. Uses the Karmarkar-Kar

Calculates partitions of indices from seqlen_list such that the sum of sequence lengths in each partition is balanced. Uses the Karmarkar-Kar

Initialize distributed training workers using Ray backend. Creates: 1. Ray resource pools from configuration 2. Worker groups

Initialize distributed training workers using Ray backend. Creates: 1. Ray resource pools from configuration 2. Worker groups

Check if padding is enabled for the DataProto. Returns: bool: True if padding is enabled, False otherwise.

Check if padding is enabled for the DataProto. Returns: bool: True if padding is enabled, False otherwise.

check if the package version is greater than or equal to the minimum version

Compute KL divergence given logprob and ref_logprob. Copied from https://github.com/huggingface/trl/blob/main/trl/trainer/ppo_trainer.py#L1104

Load a external data type based on the file path and type name

Creates a batch generator suitable for Megatron pipeline parallelism, handling virtual pipeline parallelism (VPP). If VPP is used (vpp_s

Normalize a final answer to a quantitative reasoning question. Args: final_answer: The answer string to normalize Returns: N

Normalize a final answer to a quantitative reasoning question. Args: final_answer: The answer string to normalize Returns: N

Normalize a final answer to a quantitative reasoning question. Args: final_answer: The answer string to normalize Returns: N

Pad a DataProto to size divisible by size_divisor Args: size_divisor (int): size divisor Returns: data: (DataProto): the pad

Note that this operation is in-place

if test(generated_code) is not None it'll try to run the code. otherwise it'll just return an input and output pair.

if test(generated_code) is not None it'll try to run the code. otherwise it'll just return an input and output pair.

Select specific indices from the DataProto. Args: idxs (torch.Tensor or numpy.ndarray or list): Indices to select

Select specific indices from the DataProto. Args: idxs (torch.Tensor or numpy.ndarray or list): Indices to select

Set input tensor to be used instead of forward()'s input. When doing pipeline parallelism the input from the previous stage comes fro

Set input tensor to be used instead of forward()'s input. When doing pipeline parallelism the input from the previous stage comes fro

Spawn to a dictionary of worker groups, each with a subset of method with prefix. Args: prefix_set: Set of prefixes to create wor

Decorator to add a timeout to a function. Args: seconds: The timeout duration in seconds. use_signals: (Deprecated) This is

Split along the second dim into `n_split`, unfold it to the first dim (batch dim) Useful in passing grouped tensors that doesn't want to be sh

Union two tensordicts.

Update the module config with the override_config_kwargs. Args: module_config: The module config from Huggingface Transformers. ov