A DataProto is a data structure that aims to provide a standard protocol for data exchange between functions. It contains a batch (TensorDict

Data structure for the output of reward calculations. Attributes: reward (float): The computed reward value based on the evaluation of th

Sharding manager to support data resharding when using FSDP + Ulysses

Used to count mfu during training loop Example: flops_counter = FlopsCounter(config) flops_achieved, flops_promised = flops_

Define a resource pool specification. Resource pool will be initialized first.

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save -

We assume the dataset contains a column that contains prompts and other information

Note that this trainer runs on the driver process on a single CPU/GPU node.

AsyncLLMServerManager manage a group of vllm instances, i.e AsyncvLLMServer.

DataProtoFuture aims to eliminate actual data fetching on driver. By doing so, the driver doesn't have to wait for data so that asynchronous

The implementation of a simple digit completion task. The prompt is a sequence of numbers with fixed difference. The task is to complete the

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save -

A memory buffer is a contiguous torch tensor that may combine multiple tensors sharing with the underlying memory. It must have a unique type

Dataset for multi-turn conversations where each assistant response should be trained

Data structure for input required to calculate rewards. Attributes: problem (str): The original problem text or prompt provided to the mo

The reward manager.

Reward function for evaluating mathematical answers. This class implements the __call__ method to process the input and determine the re

This is an in-memory SFTDataset Arguments: config (OmegaConf): the data config

A test class to be imported by load_extern_type

Adaptive KL controller described in the paper: https://arxiv.org/pdf/1909.08593.pdf

The data model for async rollout.

Fixed KL controller.

LlamaRotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla

LlamaRotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev

MegatronWorkerGroup will query each worker of its megatron rank info and store it inside the WorkerGroup so that the dispatcher can use it to

The parsed schema of a tool in OpenAI format.

The tool call in OpenAI format.

Multi-headed attention from 'Attention Is All You Need' paper

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] This model definition supports pip

Multi-headed attention from 'Attention Is All You Need' paper

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] Args: config: Qwen2Config

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] Args: config: Qwen2Config

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] This model definition supports pip

Note that this trainer runs on the driver process on a single CPU/GPU node.

Note that this trainer runs on the driver process on a single CPU/GPU node.

Note that this trainer runs on the driver process on a single CPU/GPU node.