broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Compute loss with optional sequence parallelism and remove padding features

Computes information about prompts and responses from a batch. This is an internal helper function that extracts masks and lengths for p

Computes information about prompts and responses from a batch. This is an internal helper function that extracts masks and lengths for p

Dump rollout/validation samples as JSONL.

Dump rollout/validation samples as JSONL.

Normalize answer expressions.

Currently, we only support chatml format.

Currently, we only support chatml format.

model: AutoModelForCausalLM

Apply KL penalty to the token-level rewards. This function computes the KL divergence between the reference policy and current policy, then a

Apply KL penalty to the token-level rewards. This function computes the KL divergence between the reference policy and current policy, then a

Performs bootstrap resampling to estimate statistics of metrics. This function uses bootstrap resampling to estimate the mean and standard d

from https://github.com/sgl-project/sglang/blob/844e2f227ab0cce6ef818a719170ce37b9eb1e1b/python/sglang/srt/utils.py#L905 Broadcast inputs from sr

Check if generated code passes all test cases within a timeout period. Args: tests: Test cases in either list of dictionaries or dic

Check if generated code passes all test cases within a timeout period. Args: tests: Test cases in either list of dictionaries or dic

Computes various metrics from a batch of data for PPO training. This function calculates metrics related to scores, rewards, advantages, ret

We compute dense reward here so that we can directly train RL without SFT

Computes timing metrics for different processing stages in PPO training. This function calculates both raw timing metrics (in seconds) a

This function should return a class instance that delegates the calls to every cls in cls_dict

Create a dataset.

Compute the score for a given solution based on the data source. Args: data_source (str): The source dataset identifier which determines

Estimate the FLOPS based on the number of valid tokens in the current batch and the time taken. Args: batch_seqlens (Lis

Execute a method on rank zero worker asynchronously. Args: method_name: Name of the method to execute *args: Position

Execute a method on rank zero worker asynchronously. Args: method_name: Name of the method to execute *args: Position

Loads the full state dict (could be only on rank 0) into the sharded model. This is done by broadcasting the parameters from rank 0 to all ot

Loads the full state dict (could be only on rank 0) into the sharded model. This is done by broadcasting the parameters from rank 0 to all ot

Create a constant LR schedule with a linear warmup phase. Args: optimizer (Optimizer): Wrapped optimizer. num_warmup_steps (

Create a schedule with a learning rate that decreases following the values of the cosine function between the initial lr set in the optimizer

Get the resource pool of the worker_cls

Get the resource pool of the worker_cls

Get the resource pool of the worker_cls

Initialize a Mcore model. Args: tfconfig: The transformer config. hf_config: The HuggingFace config. pre_process: Op

Initialize a Mcore model. Args: tfconfig: The transformer config. hf_config: The HuggingFace config. pre_process: Op

Init resource pool and worker group

Init resource pool and worker group

Extract the last LaTeX boxed expression from a string. Args: string: Input string containing LaTeX code Returns: The last bo

Load weights for mcore GPT model.

Load weights for mcore GPT model.

r"""Make an iterator from the DataProto. This is built upon that TensorDict can be used as a normal Pytorch dataset. See https://pytorch.org/t