spawn to a dictionary of worker groups, each with a subset of method with prefix.

Update the policy with an iterator of DataProto Args: data (DataProto): an iterator over the DataProto that returns by

Reorder the data on single controller such that each dp rank gets similar total tokens

Bind the worker method to the WorkerGroup

Binds the methods of each worker to the WorkerDict. Note that we only bind public methods that are decorated by register

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Build the parameter buffer in each pp rank

This function should only be called inside by WorkerGroup

hdfs capable to check whether a file_path is exists

Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.

Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.

Make causal mask used for bi-directional self-attention.

Make causal mask used for bi-directional self-attention.

Calculate the mapping of global layer_idx to local layer_idx Returns: layer_map (Dict: int -> tuple(int, int, int)): mapping f

Calculate the mapping of global layer_idx to local layer_idx Returns: layer_map (Dict: int -> tuple(int, int, int)): mapping f

hdfs mkdir

For each param, if it is a tp-splited param, we all-gather from micro_dp group.

Build the memory references. The memory buffers are built using the build_memory_buffer API. This API will allocate a weight buffer pointer to the

Check the consistency of the DataProto. Mainly for batch and non_tensor_batch We expose this function as a public one so that user can call th

Only collect the data from the tp=0 and pp=last and every dp ranks

treat pp as dp. Only collect data on tp=0

Computing reward given input_ids. The transformers should output a tensor with shape [batch_size, sequence_length], and the value at [EOS]