Puts a list of data into the Ray object store in parallel using a thread pool. Args: data_list (List[Any]): A list of Python objects

Stores a new batch of records into memory.

Submit a chat completion request to chat scheduler and wait until it is done. To submit multiple requests in parallel, please use `generate_se

Tell the agent that it's training phase.

Shape: - y_pred: batch x time x vocab - y_true: batch x time - mask: batch x time

Shape: - y_pred: batch x time x vocab - y_true: batch x time - mask: batch x time

Reorder the data on single controller such that each dp rank gets similar total tokens

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Compute loss with optional sequence parallelism and remove padding features

Computes information about prompts and responses from a batch. This is an internal helper function that extracts masks and lengths for p

Convert a single trajectory text to an image with optimized packing. Args: trajectory_text: Trajectory text stri

Get or create a cached font object to avoid repeated font loading. This provides significant speedup for repeated text rendering.

Get configuration dictionary, merging defaults with overrides. Args: **override_kwargs: Parameters to override

Returns web request result wrapped in BeautifulSoup object Arguments: url (`str`): If no url or html is provided, use the cu

Currently, we only support chatml format.

Apply KL penalty to the token-level rewards. This function computes the KL divergence between the reference policy and current policy, then a

Performs bootstrap resampling to estimate statistics of metrics. This function uses bootstrap resampling to estimate the mean and standard d

Calculates the sum of all Manhattan distances, between the boxes and their origin box targets. :param box_mapping: :return:

This function builds the text observation for the agent.

Select an action randomly.

Computes various metrics from a batch of data for PPO training. This function calculates metrics related to scores, rewards, advantages, ret

We compute dense reward here so that we can directly train RL without SFT

Computes timing metrics for different processing stages in PPO training. This function calculates both raw timing metrics (in seconds) a

This function should return a class instance that delegates the calls to every cls in cls_dict

Create a dataset.

Compute the score for a given solution based on the data source. Args: data_source (str): The source dataset identifier which determines

Execute a method on rank zero worker asynchronously. Args: method_name: Name of the method to execute *args: Position

Loads the full state dict (could be only on rank 0) into the sharded model. This is done by broadcasting the parameters from rank 0 to all ot

Create a constant LR schedule with a linear warmup phase. Args: optimizer (Optimizer): Wrapped optimizer. num_warmup_steps (

Create a schedule with a learning rate that decreases following the values of the cosine function between the initial lr set in the optimizer

Determine which page (i.e. item_page, search_results) the given URL is pointing at

Get the resource pool of the worker_cls

Initialize a Mcore model. Args: tfconfig: The transformer config. hf_config: The HuggingFace config. pre_process: Op

Init resource pool and worker group

Load weights for mcore GPT model.

r"""Make an iterator from the DataProto. This is built upon that TensorDict can be used as a normal Pytorch dataset. See https://pytorch.org/t