Update the policy with an iterator of DataProto Args: data (DataProto): an iterator over the DataProto that returns by

Compute entropy when the logits are sharded in tp ranks Args: vocab_parallel_logits: (total_nnz, vocab_size // tp_size) Returns: (to

Binds the methods of each worker to the WorkerDict. Note that we only bind public methods that are decorated by register

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Build the parameter buffer in each pp rank

Check if the resource pool can be satisfied in this ray cluster.

This function should only be called inside by WorkerGroup

Creates the train and validation dataloaders.

- megatron: base class should be MegatronWorker - fsdp: base class should be Worker

hdfs capable to check whether a file_path is exists

Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.

Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.

Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.

fetch gate_up tensor in tp shards

fetch gate_up tensor in tp shards

fetch tensor in tp shards across mp_group

fetch tensor in tp shards

fetch tensor in tp shards

Get the total CPU memory capacity in GB.

Get GPU type, GPU memory, and GPU count using nvidia-smi command.

Get CPU memory capacity, GPU type, GPU memory, and GPU count.