A DataProto is a data structure that aims to provide a standard protocol for data exchange between functions. It contains a batch (TensorDict

Sharding manager to support data resharding when using FSDP + Ulysses

Used to count mfu during training loop Example: flops_counter = FlopsCounter(config) flops_achieved, flops_promised = flops_

DataProtoFuture aims to eliminate actual data fetching on driver. By doing so, the driver doesn't have to wait for data so that asynchronous

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save

A memory buffer is a contiguous torch tensor that may combine multiple tensors sharing with the underlying memory. It must have a unique type

We assume the dataset contains a column that contains prompts and other information

This is an in-memory SFTDataset

VimRAG: Visual Memory RAG Agent with DAG-based reasoning.

Adaptive KL controller described in the paper: https://arxiv.org/pdf/1909.08593.pdf

Fixed KL controller.

LlamaRotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla

LlamaRotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev

Multi-headed attention from 'Attention Is All You Need' paper

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] This model definition supports pip

Multi-headed attention from 'Attention Is All You Need' paper

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] Args: config: Qwen2Config

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] Args: config: Qwen2Config

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] This model definition supports pip

Note that this trainer runs on the driver process on a single CPU/GPU node.

Define a resource pool specification. Resource pool will be initialized first. Mapping

This worker can be instantiated as a standalone actor or a standalone rollout or a standalone reference policy or a hybrid engine based on th

This worker can be instantiated as a standalone actor or a standalone rollout or a standalone reference policy or a hybrid engine based on th

Using an enumeration class to avoid spelling errors in adv_estimator

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save -

This class stores a class constructor and the args/kwargs to construct the class. It is used to instantiate the remote class.