Get the generation prompt ids for rollout engine. Because rollout engine(SGLang) requires the ids to be a list, we need to convert t

Get the forward function for given model architecture.

Get the forward function for given model architecture.

Get the weight converter for given model architecture.

Group tensors into buckets based on a specified size in megabytes. Args: iterable: An iterator of tuples containing tensor names and

Generate and return an NPU profiler object. Args: contents (list[str]): A list of options to control the collection content,

Determine the number of transformer layers to build for the current pipeline stage. Args: config (TransformerConfig): Configuration o

Get PEFT class from model config. Args: model_config: Model configuration object. bridge: Megatron-Bridge AutoBridge instance.

Get rollout worker actor class for colocated and standalone mode.

Gets the position ids for Qwen2-VL, it should be generated before sharding the sequence. The batch dim has been removed and the input_ids sho

Choose the dimension that can divide fsdp_size to avoid padding

Determine the appropriate sharding strategy based on the number of dimensions of the device mesh. Args: device_mesh (torch.distribut

Get statistics (sync - deprecated, use get_statistics instead)

Get ulysses sequence parallel rank.

Util to extract Vision Tower from a VL model instance

For vLLM, automatically adjusts the `max_lora_rank` to the nearest allowed value. The allowed values are retrieved from vLLM's MaxLoRARanks t

The answer will be considered correct if: (a) it normalizes to the same string as the ground truth answer OR (b) sympy can simplify t

Init colocated rollout server, rollout engine and hybrid engine colocated in same ray placement group but in separate processes. Args

Initialize distributed training workers using Ray backend. Creates: 1. Ray resource pools from configuration 2. Worker groups

Instantiate or load the model, optimizer, and learning rate scheduler. Should prepare all components necessary for training or evalu

Initialize tools from config file. Supports both NATIVE and MCP tool types. For MCP tools, a temporary event loop is created only when needed

Clear all cached weight scales after optimizer.step().

Return True if the wrapped data supports chunk dispatch.

Return True if the wrapped list of data supports concat collect.

Check if padding is enabled for the DataProto. Returns: bool: True if padding is enabled, False otherwise.

Return True if the current router_replay_action is REPLAY_BACKWARD for the local router instances. This inspects the first local RouterReplay

Convert a list of dictionaries to a dictionary of lists. Args: list_of_dict: List of dictionaries with same keys Returns:

Load an FSDP checkpoint for this rank. Downloads and loads: - model and optimizer shards - extra state dict (sch

Load merged state_dict to sharded Megatron module in training.

Load configuration from YAML file. Args: config_path: Path to YAML configuration file Returns: Dictionary with configuration

Compute the log_probs from logits with rmpad input_ids and logits. Note that logits_rmpad = model(input_ids_rmpad). For each sentences, there is a

Calculate and log metrics related to sequence length imbalance before and after partitioning. Args: seqlen_list (List[int]): A list

Make minibatch iterator for updating the actor Args: data (DataProto): a DataProto containing keys ``input_ids``

End a mark range in the profiler. Args: range_id (str): The id of the mark range to end.

Start a mark range in the profiler. Args: message (str, optional): The message to be displayed in the profiler. Defaults to N

Compute a similarity score considering element frequency, ignoring order. Reference: Liu S Y, Dong X, Lu X, et al. "Gdpo: Group reward-decoupled

Exact match of math if and only if: 1. numerical equal: both can convert to float and are equal 2. symbolic equal: both can convert to sy

Convert a MCP Tool to an OpenAI ChatCompletionTool.

Generate an MD5 hash of a path string. This function is used to create unique identifiers for paths, typically for creating cache directories

Merge recorded router top-k indices across sequence-parallel ranks for all router instances, then pack/unpack them to align with the original

Context manager to temporarily merge LoRA adapters. This context manager merges LoRA adapters into the base model weights, performs operation

Check if compute must stall for prefetch.

Given the config, do we need distillation policy.

Transform the model name in each model_chunk in each pp stage into the name in inference engine

Offload.

Offload optimizer state to CPU.

On group commit forward.

Zero the gradients of the optimizer.

Zero gradients and enforce FSDP grad-clipping logic.

pad a 2D tensors (e.g. responses, logprobs) in the last dim to max_seq_length. input shape: [bs, seq_length] output shape: [bs, max_seq_l

Pad the DataProto by concating with padding_candidate.repeat(padding_size) Args: padding_size (int): the number of repeated paddi