Check if the resource pool can be satisfied in this ray cluster.

This function should only be called inside by WorkerGroup

Patches flash attention forward to handle 3D position ids in mrope. (3, batch_size, seq_length)

Compute the model size.

Log a table of validation samples

TODO: optimize this. - We can use async ops - We can use only one allgather

Convert base64 string to PIL Image

Check the consistency of the DataProto. Mainly for batch and non_tensor_batch We expose this function as a public one so that user can call th

This function should return a class instance that delegates the calls to every cls in cls_dict

Extracts the content of the <answer>…</answer> block from `predict`. Returns the inner text (with leading/trailing whitespace stripped),

Extracts the content of the <answer>…</answer> block from `predict`. Returns the inner text (with leading/trailing whitespace stripped),

The training loop of PPO. The driver process only need to call the compute functions of the worker group through RPC to construct the

A func to sync attention result with alltoall in sequence parallel gather head dimension and scatter seq dim: e.g. seq_dim: 1, head_dim:

Tracker file rescords the latest chckpoint during training to restart from.

Get the lr scheduler for constant lr.

Get FSDP wrap policy for the model. Args: module: The module to get wrap policy for

Note that here we only asks execute_all and execute_rank_zero to be implemented Leave the choice of how these two functions handle argument '

Gets the position ids for Qwen2-VL, it should be generated before sharding the sequence. The batch dim has been removed and the input_ids sho

Get ulysses sequence parallel rank.

Convert PIL Image to base64 string

Init resource pool and worker group