Add a prefix to all the keys in the data dict if it's name is not started with prefix Args: data: a dictionary prefix: prefix

Note that this operation is in-place

Replace LoRA parameter keys with base layer equivalents. Transforms LoRA parameter names to their corresponding base layer names for proper w

Rotates half the hidden dims of the input.

Initialize Ray cluster and run distributed PPO training process. Args: config: Training configuration object containing all necessary par

Compute Q-values for given state-action pairs. Args: a: Dictionary of tensors representing actions, with key: - "f

Setup test cases with a mock registry.

Start profiling on all rollout replicas.

Stop profiling on all rollout replicas.

Split a batch into N mini-batches run for multiple epochs Args: data: Returns:

Update the Ray actor creation options. Args: options: Dictionary of options to update

Validate critic configuration parameters.

Binds worker methods to the WorkerGroup based on registered attributes. Args: user_defined_cls (type): The class containing metho

broadcast tensor in tp shards across mp_group

Check if this is the first process_weights call, and increment counter.

Cleans the merged lora adapters

Compute multi-modal inputs with image and video.

Compute position ids for multi-modal inputs.

compute reward use colocate reward model

Creates the train and validation dataloaders.

Create a Parameter from saved metadata. Used by rebuild and tensor swap.

Create test DataProto for compute_log_prob method

Execute method on inference engine via ray. Args: method: The method name to execute on the server. non_block: If Tru

Execute a method on a single worker remotely. Args: worker: The worker actor handle method_name: Name of the method t

Get current memory usage. Note that CPU device memory info is always 0. Args: unit (str, optional): The unit of memory measurement.

Traverse nested attributes. Returns None if any step fails.

Check if a node has a docstring.

Merge or unmerge LoRA adapters in a module. Args: module: The module containing LoRA layers merge: If True, merge LoRA into base

Process the padded outputs from _run_agent_loop and combine them into a batch.

Process proprioceptive features and append to vision features

Tokenize a text prompt into valid token IDs for the model.

Tokenize on the asyncio thread for fast tokenizers when no processor is used. Rust-backed fast tokenizers are not reliably safe across ``run_

Check if we can issue another prefetch.

Clears the recorded and target topk indices in all instances.

Clears the router replay action for all router instances.

Merge LoRA into base weights and extract full state dict with HF key names. For rollout backends (e.g. SGLang) whose load_weights() expects stand

Compute GRPO advantages from each session's final output. For non-GRPO estimators, such as GAE, are delegated to the original compute_advantage()

Compute per-token log probabilities from logits.

Compute rollout correction weights and apply rejection sampling. Computes importance sampling weights to correct for off-policy issues between

Compute variance proxy metrics using the simplified expected squared norm approach. This metric provides a computationally efficient way to

Create agent loop manager.

Create a simple base model for testing.

Create a device mesh for distributed training based on the world size and FSDP size. Args: world_size (int): Total number of process

Create resource pool manager Args: config: Configuration object roles: List of roles that need to create resource pools