Enable or disable expandable segments for cuda. Args: enable (bool): Whether to enable expandable segments. Used to avoid OOM.

Spawn to a dictionary of worker groups, each with a subset of method with prefix. Args: prefix_set: Set of prefixes to create wor

Start profiling for the current rank in the current training step.

Stop profiling for the current rank in the current training step.

Calculate topk items take-up ratio, and min/max timing across all ranks.

Binds worker methods to the WorkerGroup based on registered attributes. Args: user_defined_cls (type): The class containing metho

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Execute a method on a single worker remotely. Args: worker: The worker actor handle method_name: Name of the method t

fetch tensor in tp shards

fetch tensor in tp shards

Get current memory usage. Note that CPU device memory info is always 0. Args: unit (str, optional): The unit of memory measurement.

Process tokens for a single message or a group of messages. Args: messages: List of message dictionaries sta

Start profiling for all worker groups if profiling is enabled.

Stop profiling for all worker groups if profiling is enabled.

TODO: optimize this. Technically, we only need one broadcast

for cuda memory alignment, make sure alignment by 128-bits

Compute advantage estimates for policy optimization. This function computes advantage estimates using various estimators like GAE, GRPO, REINFORC

Compute logits given a batch of data. Args: data (DataProto): a batch of data represented by DataProto. It must contain key ```in

Computing reward given input_ids. The transformers should output a tensor with shape [batch_size, sequence_length], and the value at [EOS]

Computes throughput metrics for PPO training. This function calculates performance metrics related to token processing speed, including

This function should return a class instance that delegates the calls to every cls in cls_dict

Create Ray resource pools for distributed training. Initializes resource pools based on the resource pool specification, with each po

Create a dataset. Arguments: data_paths: List of paths to data files. data_config: The data config. tokenizer (Tokenizer)

Enable activation offloading for the model. It groups activations by TransformerLayer and offloads activation groups asynchronously. This mea

Return the most recent checkpoint directory based on a tracker file. Args: path (str): Base directory containing the checkpoint trac

Generate tokens from prompt ids. Args: request_id (str): request id for sticky session. prompt_ids (List[int]): List

Create a constant LR schedule with a linear warmup phase. Args: optimizer (Optimizer): Wrapped optimizer. num_warmup_steps (

Get the dataset paths for the given task configs and split. Args: task_configs: The task configs split: The dataset split Re

Get the forward function for given model architecture.

Get the number of gpus in this cluster.

Get the policy loss with a given name. Args: name: `(str)` The name of the policy loss. Returns: `(callable)`: T

A filter function to return the PPO Ray runtime environment. To avoid repeat of some environment variables that are already set.

Choose the dimension that can divide fsdp_size to avoid padding

Function that gets visible devices keyword name. Returns: 'CUDA_VISIBLE_DEVICES' or `ASCEND_RT_VISIBLE_DEVICES`

Initialize a Mcore model. Args: tfconfig: The transformer config. hf_config: The HuggingFace config. pre_process: Op

Creates a batch generator suitable for Megatron pipeline parallelism, handling virtual pipeline parallelism (VPP). If VPP is used (vpp_s

r"""Works like os.makedirs() but supports hdfs. Super-mkdir; create a leaf directory and all intermediate ones. Works like mkdir, except tha

End a mark range in the profiler. Args: range_id (str): The id of the mark range to end.

Start a mark range in the profiler. Args: message (str, optional): The message to be displayed in the profiler. Defaults to N

Normalize a final answer to a quantitative reasoning question. Args: final_answer: The answer string to normalize Returns: N

Release the tool instance. Args: instance_id: The instance id of the tool.

Restore a batch from dynamic batching. Args: data (torch.Tensor): The input data. batch_idx_list (List[List[int]]): The list

Save checkpoint using FSDPCheckpointManager with improved tracking

Decorator to add a timeout to a function. Args: seconds: The timeout duration in seconds. use_signals: (Deprecated) This is

Move FSDP model and/or optimizer to CPU or GPU with offload support.

Update the module config with the override_config_kwargs. Args: module_config: The module config from Huggingface Transformers. ov

Update the Ray actor creation options. Args: options: Dictionary of options to update

Validate actor configuration with runtime parameters.

Verify if the solution is correct. Args: solution_str: The solution string to verify answer: The ground truth answer stri