A DataProto is a data structure that aims to provide a standard protocol for data exchange between functions. It contains a batch (TensorDict

Sharding manager to support data resharding when using FSDP + Ulysses

Used to count mfu during training loop Example: flops_counter = FlopsCounter(config) flops_achieved, flops_promised = flops_

We assume the dataset contains a column that contains prompts and other information

Define a resource pool specification. Resource pool will be initialized first. Mapping

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save

Note that this trainer runs on the driver process on a single CPU/GPU node.

This is an in-memory SFTDataset

DataProtoFuture aims to eliminate actual data fetching on driver. By doing so, the driver doesn't have to wait for data so that asynchronous

A memory buffer is a contiguous torch tensor that may combine multiple tensors sharing with the underlying memory. It must have a unique type

Adaptive KL controller described in the paper: https://arxiv.org/pdf/1909.08593.pdf

The implementation of a simple digit completion task. The prompt is a sequence of numbers with fixed difference. The task is to complete the

Fixed KL controller.

LlamaRotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla

LlamaRotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev

Note that we do not design MemoryBufferModuleWrapper as an nn.Module due to - It will change the checkpoint name

MegatronWorkerGroup will query each worker of its megatron rank info and store it inside the WorkerGroup so that the dispatcher can use it to

Multi-headed attention from 'Attention Is All You Need' paper

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] This model definition supports pip

Multi-headed attention from 'Attention Is All You Need' paper

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] Args: config: Qwen2Config

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] Args: config: Qwen2Config

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] This model definition supports pip

Note that this trainer runs on the driver process on a single CPU/GPU node.

This worker can be instantiated as a standalone actor or a standalone rollout or a standalone reference policy or a hybrid engine based on th

This worker can be instantiated as a standalone actor or a standalone rollout or a standalone reference policy or a hybrid engine based on th

Using an enumeration class to avoid spelling errors in adv_estimator

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save -