Start profiling on all rollout replicas.

Union with another DataProto. Union batch and meta_info separately. Throw an error if - there are conflict keys in batch and they are

broadcast tensor from rank0 across mp_group

Make an async HTTP request with retry logic and consistent error handling. Args: endpoint (str): The API endpoint to call (withou

Calculate global rank with support for CP/EP parallelism

computer the angle (0~2pi) between a unit vector and positive x axis

Compute reward for a batch of data. Args: data: DataProto object containing the input data. reward_fn: Reward function to com

* HTML转义

A func to sync attention result with alltoall in sequence parallel gather head dimension and scatter seq dim: e.g. seq_dim: 1, head_dim:

Split input batch and dispatch to env loop workers. Args: prompts (DataProto): Input batch. Returns: DataPro

Build the inverse of an index mapping. Args: idx_map (Sequence[int]): Sequence where idx_map[i] = j. Returns: List[int]

Convert TensorDict from left-right padding to no-padding format. Args: data: TensorDict with "input_ids", "attention_mask", "respons

This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he

Split a RayResourcePool into multiple SubRayResourcePool. resouce_pool can also be a SubRayResourcePool (have been splited) for multiple-time

Start profiling on the replica.

Stop profiling on the replica.

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

Computes various metrics from a batch of data for PPO training. This function calculates metrics related to scores, rewards, advantages, ret

计算a◎b对应的复数平方

Create Ray resource pools for distributed training. Initializes resource pools based on the resource pool specification, with each po

Remove old checkpoints to make room for a new one, keeping a safety buffer. With max_ckpt_to_keep=1, this does nothing - we keep the

Initialize interactions from configuration file. Args: interaction_config_file: Path to the interaction configuration file. Returns:

Creates a batch generator suitable for Megatron pipeline parallelism, handling virtual pipeline parallelism (VPP). If VPP is used (vpp_s

r"""Make an iterator from the DataProto. This is built upon that TensorDict can be used as a normal Pytorch dataset. See https://pytorch.org/t

Get the rank of this worker in the distributed setup.

Restore a batch from dynamic batching. Args: data (torch.Tensor): The input data. batch_idx_list (List[List[int]]): The list

运行服务器

先转字符串，再做日志脱敏，适合打印 dict / list / 任意对象。

Configure CUDA memory allocator expandable segments setting. Expandable segments can help avoid out-of-memory (OOM) errors by allowing the me

Context manager for basic timing without NVTX markers. This utility function measures the execution time of code within its context and accum

Sleep all rollout replicas: free weight and kv_cache device memory.

Move FSDP model and/or optimizer to CPU or GPU with offload support. Note that this function executes irrespective of offload config.

given TP,DP,PP rank to get the global rank.

given TP,DP,PP rank to get the global rank.

model: AutoModelForCausalLM

Apply monkey patch to the models for ulysses sequence parallel, fused kernel, tiled MLP and prefix grouper. In the end of this function forw

Computes throughput metrics for PPO training. This function calculates performance metrics related to token processing speed, including

Computes timing metrics for different processing stages in PPO training. This function calculates both raw timing metrics (in seconds) and p

This function should return a class instance that delegates the calls to every cls in cls_dict

Converts a string to a sequence of ids (integer), using the tokenizer and vocabulary. Args: text (`str`, `List[str]` or

Estimate the FLOPS based on the number of valid tokens in the current batch and the time taken. Args: batch_seqlens (Lis

Extract and process multi-modal inputs from a batch. Args: batch_data (list[dict[str, torch.Tensor]]): The batch containing potentia

Finalizes the interaction session and releases any associated state or resources. Simulates: release state

Create a DataProto from a TensorDict. This assumes that 1. All the tensor in tensor_dict have the same dim0 2. Only dim0 is the batch

Split input batch and dispatch to agent loop workers. Args: prompts (DataProto): Input batch. Returns: DataP

Get the checkpoint engine class. Args: backend: The backend of the checkpoint engine. Returns: The checkpoin

Return the list of RouterReplay instances corresponding to the current micro-batch and local (pp_rank, vp_stage) layer range.

Get the reward manager class with a given name. Args: name: `(str)` The name of the reward manager. Returns: `(t