broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Get processed responses from LLM. Args: prompts: the prompts n: repeated num for each prompt answ

Get processed responses from VLM with image inputs. Args: prompts: the prompts images: image paths (one per p

Dynamically import attention functions based on available hardware.

Inner function that handles the core timing logic. Args: name (str): The name/identifier for this timing measurement. timing_raw

Update the input_ids, attention_mask, position_ids, and loss_mask of the request in additive manner.

Build an optimizer based on the configuration. Dynamically imports and instantiates an optimizer class from the specified module. Args:

Create a dataset. Arguments: data_paths: List of paths to data files. data_config: The data config. tokenizer (Tokenizer)

Evaluate, filter, and split samples from dataset. Args: dataset: Dataset, samples with 'input' and 'output' keys (and 'i

Extract and process multi-modal inputs from a batch. Args: batch_data (list[dict[str, torch.Tensor]]): The batch containing potentia

Format data as SSE event string.

torch.nn.utils.clip_grad_norm_ cann't run on cpu parameter DTensor

Create a schedule with a learning rate that decreases following the values of the cosine function between the initial lr set in the optimizer

Determine the number of transformer layers to build for the current pipeline stage. Args: config (TransformerConfig): Configuration o

Convert huggingface PretrainedConfig to mcore TransformerConfig. Args: hf_config: The huggingface PretrainedConfig. dtype: The dt

Creates a batch generator suitable for Megatron pipeline parallelism, handling virtual pipeline parallelism (VPP). If VPP is used (vpp_s

r"""Make an iterator from the DataProto. This is built upon that TensorDict can be used as a normal Pytorch dataset. See https://pytorch.org/t

Convert NestedTensor from no-padding to right padding format. Args: nested_tensor: NestedTensor with no-padding format data:

pad a 2D tensors (e.g. responses, logprobs) in the last dim to max_seq_length. input shape: [bs, seq_length] output shape: [bs, max_seq_l

Parse raw LLM response strings and validate with majority voting. This is equivalent to Data_Synthesis_RL's precise_check function w

Prepare a batch for dynamic batching. Args: data (DataProto): The input data. max_token_len (int): The maximum token length

Reset the model engine to the initial state. If the engine is not initialized, we initialize it. Otherwise, reload ckpt and reset sta

Restore a batch from dynamic batching. Args: data (torch.Tensor): The input data. batch_idx_list (List[List[int]]): The list

Rotates half the hidden dims of the input.

Rotates half the hidden dims of the input.

Start profiling for the current rank in the current training step.

Stop profiling for the current rank in the current training step.

Manual control of load/offload

Update the module config with the override_config_kwargs. Args: module_config: The module config from Huggingface Transformers. ov

Update the weights of the rollout model. Args: weights: A generator that yields the name of the weight tensor and the tensor itse

Validate actor configuration with runtime parameters.

Validate SFT configuration. Args: config: SFT configuration to validate Returns: Tuple of (is_valid

Binds worker methods to the WorkerGroup based on registered attributes. Args: user_defined_cls (type): The class containing metho

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Execute a method on a single worker remotely. Args: worker: The worker actor handle method_name: Name of the method t

fetch tensor in tp shards

fetch tensor in tp shards

Get current memory usage. Note that CPU device memory info is always 0. Args: unit (str, optional): The unit of memory measurement.

Load vLLM model on first use (lazy loading). This method is called automatically when the model is first needed. Uses the GP

Parse LLM response to extract list of {'input': ..., 'output': ...}. Args: response: Raw LLM response string Re

Read samples from a JSONL file. Args: jsonl_path: Path to JSONL file Returns: List of sample dictionaries

Save a single uploaded file to the target path.

Save samples to Parquet file. Args: samples: List of sample dictionaries output_path: Output Parquet file path Returns:

Slice response from unpad model output. Args: tensor: model output tensor of shape [bsz, 1] data: TensorDict with "prompt_ids", "

TODO: optimize this. Technically, we only need one broadcast

for cuda memory alignment, make sure alignment by 128-bits

Mark job as cancelled.

Validate mutually exclusive micro batch size configuration options. Ensures that users don't set both deprecated micro_batch_size and

collect lora params or full params if base model is not ready in vllm work with if isinstance(self.module._fsdp_wrapped_module, PeftModel)

Compute reward for a batch of data. Args: data: DataProto object containing the input data. reward_fn: Reward function to com