Normalize answer expressions.

Currently, we only support chatml format.

Replace _flash_attention_forward to _ulysses_flash_attention_forward

Compute autocorrelation at specific lag with better error handling

TODO: optimize this. Technically, we only need one broadcast

r"""Works like shutil.copy() for file, and shutil.copytree for dir, and supports hdfs. Copy data and mode bits ("cp src dst"). Return the file's

Create a dataset. Arguments: data_config: The data config. tokenizer (Tokenizer): The tokenizer. processor (Processor): T

Estimate the FLOPS based on the number of valid tokens in the current batch and the time taken. Args: batch_seqlens (Lis

Create a DataProto from a dict of tensors and non_tensors

Generate sequences

Get FSDP wrap policy for the module. Args: module: The module to get wrap policy for config: Configuration for wrap policy

Get the resource pool of the worker_cls

Create a huggingface processor to process multimodal data. Args: name_or_path (str): The name of the processor. Returns: tra

Compute KL divergence given logprob and ref_logprob. Copied from https://github.com/huggingface/trl/blob/main/trl/trainer/ppo_trainer.py#L1104

Load weights for mcore GPT model.

Reduces a dictionary of metric lists by computing the mean of each list. Args: metrics: A dictionary mapping metric names to lists o

Rotates half the hidden dims of the input.

Rotates half the hidden dims of the input.

Bind the worker method to the WorkerGroup

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

fetch tensor in tp shards

fetch tensor in tp shards

Get current memory usage.

model: AutoModelForCausalLM

Filters logits using adaptive probability thresholding.

Build the memory buffer given weight_buffer_meta Args: weight_buffer_meta: contains mapping from name to a dictionary containing shape an

for cuda memory alignment, make sure alignment by 128-bits

Compute comprehensive statistics for entropy sequences using vectorized operations with chunked processing. Args: entropy_tensor

Loads the full state dict (could be only on rank 0) into the sharded model. This is done by broadcasting the parameters from rank 0 to all ot

Get the forward function for given model architecture.

Initialize a Mcore model. Args: tfconfig: The transformer config. hf_config: The HuggingFace config. pre_process: Op

Check if padding is enabled for the DataProto. Returns: bool: True if padding is enabled, False otherwise.

Load a external data type based on the file path and type name

Normalize a final answer to a quantitative reasoning question. Args: final_answer: The answer string to normalize Returns: N

Transform the model name in each model_chunk in each pp stage into the name in inference engine

Pad a DataProto to size divisible by size_divisor Args: size_divisor (int): size divisor Returns: data: (DataProto): the pad

Select specific indices from the DataProto. Args: idxs (torch.Tensor or numpy.ndarray or list): Indices to select

Set input tensor to be used instead of forward()'s input. When doing pipeline parallelism the input from the previous stage comes fro