Prepare micro batches from data.

AllGather data from sp region This is because the data is first sharded along the FSDP dimension as we utilize the DP_COMPUTE

Preprocess packed sequences CP splits sequence into CP*2 chunks, and each GPU gets 2 chunks (GPU0 gets first and last chunks, GPU1 gets s

Converts a video dict into a [n_frames, 3, H, W] tensor Add video sample FPS in a future MR

Put information dictionary and extra lines on an image. Args: image: Input image info: Dictionary of key-value pairs to disp

Put a batch sample into the queue Args: sample: Sample data param_version: Parameter version number

Register a new rollout replica type.

Release GPU memory occupation temporarily. Args: tags (Optional[List[str]], optional): List of tags to specify which memory to re

Reload.

Note that this function only rename the key in the batch

Repeat the batch dict a specified number of times. Args: repeat_times (int): Number of times to repeat the data.

This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he

This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he

This is the equivalent of torch.repeat_interleave(x, dim=2, repeats=n_rep). The hidden states go from (batch, seqlen, num_key_value_heads, he

Takes numpy array corresponding to a single image and returns resized image as numpy array. Args: img: Input image as numpy array

Context switch hybridengine to rollout mode.

Context switch hybridengine to rollout mode.

Rotates half the hidden dims of the input.

Returns the closest integer to 'number' that is divisible by 'factor'.

Main entrance, supports interruption/recovery Args: sampling_params: Sampling parameters cancellation_event:

Execute the main PPO training workflow. This method sets up the distributed training environment, initializes workers, datasets, and

Initialize Ray cluster and run distributed PPO training process. Args: config: Training configuration object containing all necessary par

if test(generated_code) is not None it'll try to run the code. otherwise it'll just return an input and output pair.

Repeat each row of the batch data a specified number of times. Args: repeat_times (torch.tensor, list, tuple, ndarray):

Save an FSDP checkpoint for this rank. Writes: - model & optimizer shard files - extra state dict (scheduler + R

Saves an MP4 replay of an episode. Args: rollout_images: List of images from the episode output_dir: Directory to save the v

Shutdown the message queue

Sort the placement groups by node ip, all bundles in a single placement group should be on the same node. FSDPCheckpointManager saves sharde

Split the elements in a tuple/interval, while handling well-formatted commas in large numbers

Split the elements in a tuple/interval, while handling well-formatted commas in large numbers

Create a tool instance. Args: instance_id: The instance id of the tool. Returns: The instance id of the tool

vLLM provides `StatelessProcessGroup` to create a process group without considering the global process group in torch.distributed. It is

Copied from maniskill https://github.com/haosulab/ManiSkill Tile multiple images to a single image comprised of nrows and an appropriate

Validate critic configuration with runtime parameters. Args: n_gpus: Total number of GPUs available train_batch_size: