load all model params to cuda

Compute the log_probs from logits with rmpad input_ids and logits. Note that logits_rmpad = model(input_ids_rmpad). For each sentences, there is a

Create a mapping function to process dataset examples. Args: split: Dataset split name ('train' or 'test') Returns: Function

Create a mapping function to process dataset examples. Args: split: Dataset split name ('train' or 'test') Returns: Function

Make minibatch iterator for updating the actor Args: data (DataProto): a DataProto containing keys ``input_ids``

Normalize the pp vpp params into a complete named parameters. This is useful when gather parameters from pp ranks and passed to a model with

AllGather data from sp region This is because the data is first sharded along the FSDP dimension as we utilize the DP_COMPUTE

spawn to a dictionary of worker groups, each with a subset of method with prefix.

Update the policy with an iterator of DataProto Args: data (DataProto): an iterator over the DataProto that returns by

Reorder the data on single controller such that each dp rank gets similar total tokens

Bind the worker method to the WorkerGroup

Binds the methods of each worker to the WorkerDict. Note that we only bind public methods that are decorated by register

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Build the parameter buffer in each pp rank

This function should only be called inside by WorkerGroup

hdfs capable to check whether a file_path is exists

Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.