broadcast tensor in tp shards across mp_group

Compute loss with optional sequence parallelism and remove padding features

TODO: optimize this. - We can use async ops - We can use only one allgather Args: tensors: size: group:

We compute dense reward here so that we can directly train RL without SFT

Estimate the FLOPS based on the number of valid tokens in the current batch and the time taken. Args: batch_seqlens (Lis

Get the resource pool of the worker_cls

r"""Make an iterator from the DataProto. This is built upon that TensorDict can be used as a normal Pytorch dataset. See https://pytorch.org/t

Pad a DataProto to size divisible by size_divisor Args: size_divisor (int): size divisor Returns: data: (DataProto): the pad

Note that this operation is in-place

Rotates half the hidden dims of the input.

Rotates half the hidden dims of the input.

Set ulysses sequence parallel process group.

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Build the memory buffer given weight_buffer_meta Args: weight_buffer_meta: contains mapping from name to a dictionary containing shape an

for cuda memory alignment, make sure alignment by 128-bits

Compute logits given a batch of data. Args: data (DataProto): a batch of data represented by DataProto. It must contain key ```in

r"""Works like shutil.copy() for file, and shutil.copytree for dir, and supports hdfs. Copy data and mode bits ("cp src dst"). Return the file's

The training loop of PPO. The driver process only need to call the compute functions of the worker group through RPC to construct the

Create a schedule with a learning rate that decreases following the values of the cosine function between the initial lr set in the optimizer

Create a huggingface processor to process multimodal data. Args: name_or_path (str): The name of the processor. Returns: tra

Set input tensor to be used instead of forward()'s input. When doing pipeline parallelism the input from the previous stage comes fro

Union two tensordicts.

TODO(zhangchi.usc1992): We may change the implementation later

Reorder the data on single controller such that each dp rank gets similar total tokens

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Returns: entropy: # (bs, response_len) log_probs: # (bs, response_len)

Normalize answer expressions.