Build the memory references. The memory buffers are built using the build_memory_buffer API. This API will allocate a weight buffer pointer to the

Calculate pass@k metric for boolean pass indicators. Pass@k is the probability that at least one sample in a random subset of k samples

Calculate the reward of the tool. Args: instance_id: The instance id of the tool. Returns: The reward of the

Checks the correctness of code generation using the remote sandbox API, processing test cases concurrently. Args: sandbox_fusion

Only collect the data from the tp=0 and pp=last and every dp ranks

treat pp as dp. Only collect data on tp=0

Adapted from https://github.com/huggingface/trl/blob/main/trl/trainer/ppo_trainer.py#L1122 Args: old_log_prob: `(torch.Tensor)`

Compute the clipped policy objective and related metrics for GSPO. See https://arxiv.org/pdf/2507.18071 for more details. Args:

Computing reward given input_ids. The transformers should output a tensor with shape [batch_size, sequence_length], and the value at [EOS]

Convert Hydra configs and other special types to regular Python types.

This function should return a class instance that delegates the calls to every cls in cls_dict

This function returns a FusedWorker class. `FusedWorker.{class_name}` -> FusedClass Use `class_name` as a param to directly access t

Args: model_name: override_config_kwargs: Returns:

Create a sampler for the dataset. Arguments: data_config: The data config. dataset (Dataset): The dataset. Returns:

name: name of the parameter train_params: training parameters infer_params (Iterable[torch.Tensor]): a iterator towards list of parameter

User passes in dp data. The data is dispatched to all tp/pp ranks with the same dp

treat pp as dp.

Execute the tool. Args: instance_id: The instance id of the tool. parameters: The json string of the parameters of th

Extract the answer from inside a LaTeX \\boxed{} command

The training loop of PPO. The driver process only need to call the compute functions of the worker group through RPC to const

The training loop of PPO. The driver process only need to call the compute functions of the worker group through RPC to const

Patches flash attention forward to handle 3D position ids in mrope. (3, batch_size, seq_length)

We assume: - The model takes input: (input_ids, attention_mask, position_ids). No rmpad for the input - The communication sha

Gather the label from data. The value in label should be [0, vocab_size) Args: data: (..., vocab_size) label (torch.IntTensor) :

Tracker file rescords the latest chckpoint during training to restart from.

Generate a unique local cache path for an HDFS resource. Creates a MD5-hashed subdirectory in cache_dir to avoid name conflicts, then returns

Note that here we only asks execute_all and execute_rank_zero to be implemented Leave the choice of how these two functions handle argument '

collect rng state across data parallel ranks

Gets the position ids for Qwen2-VL, it should be generated before sharding the sequence. The batch dim has been removed and the input_ids sho

Get rope scaling args.

Get the transformer layer specification. https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/models/gpt/gpt_layer_specs.py

Get ulysses sequence parallel rank.

Create a Warmup-Stable-Decay learning rate scheduler. The schedule follows three phases: 1. Warmup: Learning rate increases linearly fro

Initialize worker engine.

Check if the solution is correct according to Minerva criteria. Args: solution_str: The solution string to check gt: The ground t

Check if the prediction is correct using strict boxed answer criteria. Args: pred: The prediction string gt: The ground truth ans