This function should return a class instance that delegates the calls to every cls in cls_dict

Estimate the FLOPS based on the number of valid tokens in the current batch and the time taken. Args: batch_seqlens (Lis

Args: input_ids: input ids. shape (batch_size, seq_length) attention_mask: attention_mask. shape (batch_size, seq_le

We assume: - The model takes input: (input_ids, attention_mask, position_ids). No rmpad for the input - The communication sha

A func to sync attention result with alltoall in sequence parallel gather head dimension and scatter seq dim: e.g. seq_dim: 1, head_dim:

Get all the parameters of the models in all pp ranks Returns: params: List[List[Dict[str, Tensor]]]: a list of parameters in all

Create a schedule with a learning rate that decreases following the values of the cosine function between the initial lr set in the optimizer

e.g. end of sentence token=1 response_id: [0, 0, 2, 42, 3, 5, 1, 0, 0] eos_mask: [1, 1, 1, 1, 1, 1, 1, 0, 0]

get order of seq lengths to make partitions balanced, this is used in balacing sum of seqlength across dp ranks and microbatches Paramete

Return world size for the tensor model parallel group.

Return a dictionary containing name to a shape and dtype.

NOTE: This method is a hack from the open-sourced version without asertion of world_size = tp * pp Initialize model parallel groups.

NOTE: This method is a hack from the open-sourced version without asertion of world_size = tp * pp Initialize model parallel groups.

load all model params to cuda

Compute the log_probs from logits with rmpad input_ids and logits. Note that logits_rmpad = model(input_ids_rmpad). For each sentences, there is a

Make minibatch iterator for updating the actor Args: data (DataProto): a DataProto containing keys ``input_ids``

Normalize the pp vpp params into a complete named parameters. This is useful when gather parameters from pp ranks and passed to a model with

Check if the tokenizer group is alive.

AllGather data from sp region This is because the data is first sharded along the FSDP dimension as we utilize the DP_COMPUTE

spawn to a dictionary of worker groups, each with a subset of method with prefix.

Update the policy with an iterator of DataProto Args: data (DataProto): an iterator over the DataProto that returns by

Reorder the data on single controller such that each dp rank gets similar total tokens

Bind the worker method to the WorkerGroup

Binds the methods of each worker to the WorkerDict. Note that we only bind public methods that are decorated by register

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Build the parameter buffer in each pp rank