Look up task output in cache

Tell the agent that it's training phase.

Shape: - y_pred: batch x time x vocab - y_true: batch x time - mask: batch x time

Shape: - y_pred: batch x time x vocab - y_true: batch x time - mask: batch x time

broadcast tensor across mp_group

broadcast tensor across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Compute loss with optional sequence parallelism and remove padding features

Returns web request result wrapped in BeautifulSoup object Arguments: url (`str`): If no url or html is provided, use the cu

TODO: optimize this. - We can use async ops - We can use only one allgather Args: tensors: size: group:

Select an action randomly.

We compute dense reward here so that we can directly train RL without SFT

Estimate the FLOPS based on the number of valid tokens in the current batch and the time taken. Args: batch_seqlens (Lis

A func to sync attention result with alltoall in sequence parallel gather head dimension and scatter seq dim: e.g. seq_dim: 1, head_dim:

Get FSDP wrap policy for the module. Args: module: The module to get wrap policy for config: Configuration for wrap policy

Get the progress to the goal.

Get the maximum number of variations for the tasks.