A DataProto is a data structure that aims to provide a standard protocol for data exchange between functions. It contains a batch (TensorDict

Sharding manager to support data resharding when using FSDP + Ulysses

Used to count mfu during training loop Example: flops_counter = FlopsCounter(config) flops_achieved, flops_promised = flops_

DataProtoFuture aims to eliminate actual data fetching on driver. By doing so, the driver doesn't have to wait for data so that asynchronous

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save -

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save -

A memory buffer is a contiguous torch tensor that may combine multiple tensors sharing with the underlying memory. It must have a unique type

Define a resource pool specification. Resource pool will be initialized first.

Adaptive KL controller described in the paper: https://arxiv.org/pdf/1909.08593.pdf

AsyncLLMServerManager manage a group of vllm instances, i.e AsyncvLLMServer.

The data model for async rollout.

Fixed KL controller.

LlamaRotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla

LlamaRotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev

The parsed schema of a tool in OpenAI format.

The parsed schema of a tool in OpenAI format.

The tool call in OpenAI format.

Multi-headed attention from 'Attention Is All You Need' paper

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] This model definition supports pip

Multi-headed attention from 'Attention Is All You Need' paper

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] Args: config: Qwen2Config

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] Args: config: Qwen2Config

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] This model definition supports pip

Note that this trainer runs on the driver process on a single CPU/GPU node.

Note that this trainer runs on the driver process on a single CPU/GPU node.

This worker can be instantiated as a standalone actor or a standalone rollout or a standalone reference policy or a hybrid engine based on th

This worker can be instantiated as a standalone actor or a standalone rollout or a standalone reference policy or a hybrid engine based on th

Using an enumeration class to avoid spelling errors in adv_estimator

The enum for async rollout request state.

Base class for AsyncServer.

AsyncvLLMServer is a wrapper for AsyncLLM, it uses ExternalRayDistributedExecutor to launch engines in hybrid rollout workers, i.e AsyncActor

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save -

Base class for model initializers.

Base class for tools. A tool should support the following methods: - `to_openai_function_tool_schema`: return the tool schema in OpenAI form