Get trajectory info. Args: step (int): global steps in the trainer. index (list): form datastore extra_info.index column.

Util to extract Vision Tower from a VL model instance

For vLLM, the smallest `max_lora_rank` is 8, and allowed values are (8, 16, 32, 64, 128, 256, 320, 512) This function automatically adjusts t

The answer will be considered correct if: (a) it normalizes to the same string as the ground truth answer OR (b) sympy can simplify t

Compute per-group mean/std/count in pure PyTorch. mean_g = sum / count std_g = sqrt( max( (sum2 - sum^2/count) / max(count-1, 1), eps )

Perform inference on a batch of data. Args: data: The input data for inference, typically containing tensors and metadat

Initialize a Mcore model. Args: tfconfig: The transformer config. hf_config: The HuggingFace config. pre_process: Op

Instantiate or load the model, optimizer, and learning rate scheduler. Should prepare all components necessary for training or evalu

Launch http server in each node.

Load and initialize a reward manager based on the configuration. Args: config: PPO trainer configuration object containing reward_mo

Advance the learning rate scheduler by one step. Returns: current_lr (float or list[float]): Updated learning rate(s).

End a mark range in the profiler. Args: range_id (str): The id of the mark range to end.

Start a mark range in the profiler. Args: message (str, optional): The message to be displayed in the profiler. Defaults to N

Compute mean of tensor with a masked values.

Convert NestedTensor from no-padding to right padding format. Args: nested_tensor: NestedTensor with no-padding format data:

Decorator to register a advantage estimator function with a given name. Args: name_or_enum: `(str)` or `(AdvantageEstimator)`

Release weights and kv cache in GPU memory.

Return one response generator based on the received messages. This method performs a uniform type conversion for the inputted messages,

Select specific indices from the DataProto. Args: idxs (torch.Tensor or numpy.ndarray or list): Indices to select

Sleep each rollout server.

Perform a training step on a batch of data. Args: data: The input data for training, typically containing tensors and me

Context manager entry for switching the engine and model into training mode. Usage: with engine.train_mode():

Context switch hybridengine to trainer mode.

Context switch hybridengine to trainer mode.

Split along the second dim into `n_split`, unfold it to the first dim (batch dim) Useful in passing grouped tensors that doesn't want to be sh

Union two tensordicts.

Update the Ray actor creation options. Args: options: Dictionary of options to update

Update the policy with an iterator of DataProto Args: data (DataProto): an iterator over the DataProto that returns by

Compute entropy when the logits are sharded in tp ranks Args: vocab_parallel_logits: (total_nnz, vocab_size // tp_size) Returns: (to

TODO(zhangchi.usc1992): We may change the implementation later

Wake up all rollout replica instances.

Validate actor configuration parameters.

Set the value of an attribute. Check if the attr is mutable before setting the value.

Perform post-processing operations on the output of each individual agent loop.

Call parameter synchronization and asynchronous sequence generation.

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Validate and resolve the unnormalization key for action statistics

Recursively compare two OmegaConf configs and assert they are identical. Args: legacy_allow_missing (bool): sometimes the legacy