Generate sequences

Get FSDP wrap policy for the module. Args: module: The module to get wrap policy for config: Configuration for wrap policy

Get the resource pool of the worker_cls

渲染错误卡片

Rotates half the hidden dims of the input.

Rotates half the hidden dims of the input.

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Update right side state.

Build the memory buffer given weight_buffer_meta Args: weight_buffer_meta: contains mapping from name to a dictionary containing shape an

for cuda memory alignment, make sure alignment by 128-bits

r"""Works like shutil.copy() for file, and shutil.copytree for dir, and supports hdfs. Copy data and mode bits ("cp src dst"). Return the file's

Create position ids from attention mask.

Create a huggingface processor to process multimodal data. Args: name_or_path (str): The name of the processor. Returns: tra

Set input tensor to be used instead of forward()'s input. When doing pipeline parallelism the input from the previous stage comes fro

TODO(zhangchi.usc1992): We may change the implementation later

Tokenize a batch of responses.

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Compute loss with optional sequence parallelism and remove padding features

Pad responses for non-active examples with pad tokens.

Returns: entropy: # (bs, response_len) log_probs: # (bs, response_len)

Wrapper for generation that handles multi-GPU padding requirements. if num_gpus <= 1, return self.actor_rollout_wg.generate_s

Normalize answer expressions.

Process responses to stop at search operation or answer operation.

Process image to base64 string (in-memory, no disk I/O).

The training loop of PPO with global metric computation. Accumulates metrics across all batches before computing final statistics.

allgather params of all pp ranks. Return a list of handles

Cut tensors to their effective length based on attention mask.

计算折扣累积增益（DCG） :param relevance_scores: 一个列表，表示每个文档的相关性分数 :return: DCG 值

Execute predictions across multiple environments. NOTE: the function is the actual `step` function in the environment NOTE pe

Args: input_ids: input ids. shape (batch_size, seq_length) attention_mask: attention_mask. shape (batch_size, seq_le

Args: input_ids: input ids. shape (batch_size, seq_length) attention_mask: attention_mask. shape (batch_size, seq_le

We assume: - The model takes input: (input_ids, attention_mask, position_ids). No rmpad for the input - The communication sha

Get all the parameters of the models in all pp ranks Returns: params: List[List[Dict[str, Tensor]]]: a list of parameters in all