Decorator to add a timeout to a function. Args: seconds: The timeout duration in seconds. use_signals: (Deprecated) This is

Update the module config with the override_config_kwargs. Args: module_config: The module config from Huggingface Transformers. ov

TODO(zhangchi.usc1992): We may change the implementation later

Reorder the data on single controller such that each dp rank gets similar total tokens

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Compute loss with optional sequence parallelism and remove padding features

Computes information about prompts and responses from a batch. This is an internal helper function that extracts masks and lengths for p

Dump rollout/validation samples as JSONL.

fetch tensor in tp shards across mp_group

Helper function containing the loading hf model logic

Normalize answer expressions.

Performs bootstrap resampling to estimate statistics of metrics. This function uses bootstrap resampling to estimate the mean and standard d

Calls the remote sandbox API to execute code with retry logic for Gateway Timeout, using increasing delay between retries. Logs internal call

Check the consistency of the DataProto. Mainly for batch and non_tensor_batch We expose this function as a public one so that user can call th

Computes various metrics from a batch of data for PPO training. This function calculates metrics related to scores, rewards, advantages, ret

Compute logits given a batch of data. Args: data (DataProto): a batch of data represented by DataProto. It must contain key ```in

Adapted from https://github.com/huggingface/trl/blob/main/trl/trainer/ppo_trainer.py#L1122 Args: old_log_prob: `(torch.Tensor)`

Compute reward for a batch of data. Args: data: DataProto object containing the input data. reward_fn: Reward function to com

Computes throughput metrics for PPO training. This function calculates performance metrics related to token processing speed, includ

Computes timing metrics for different processing stages in PPO training. This function calculates both raw timing metrics (in seconds) a

Convert megatron model to transformers model.

Recursively convert a data structure to JSON-serializable format. - Convert numpy arrays and torch tensors to lists. - Skip u

Deprecated. Please use copy_to_local instead.

Create a dataset. Arguments: data_config: The data config. tokenizer (Tokenizer): The tokenizer. processor (Processor): T

Create a sampler for the dataset. Arguments: data_config: The data config. dataset (Dataset): The dataset. Returns:

Create a dataset.

name: name of the parameter param: training parameters infer_params (Iterable[torch.Tensor]): a iterator towards list of para

The keys of the environment variables that are used to configure the Worker.

Args: input_ids: input ids. shape (batch_size, seq_length) attention_mask: attention_mask. shape (batch_size, seq_le

Args: input_ids: input ids. shape (batch_size, seq_length) attention_mask: attention_mask. shape (batch_size, seq_le

We assume: - The model takes input: (input_ids, attention_mask, position_ids). No rmpad for the input - The communication sha

torch.nn.utils.clip_grad_norm_ cann't run on cpu parameter DTensor

Determine the directory name for this rank's checkpoint.

Create a schedule with a learning rate that decreases following the values of the cosine function between the initial lr set in the optimizer

Get the weight converter for given model architecture.

Get the number of gpus in this cluster.

get order of seq lengths to make partitions balanced, this is used in balacing sum of seqlength across dp ranks and microbatches Parameter

Return a dictionary containing name to a shape and dtype.

The answer will be considered correct if: (a) it normalizes to the same string as the ground truth answer OR (b) sympy can simplify t

Init resource pool and worker group

Check if the given string is an IPv6 address Args: ip_str: The IP address string to check Returns: bool: Returns True i