A wrapper class for Ray actors with initialization arguments. This class extends ClassWithInitArgs to provide additional functionality for co

A DataProto is a data structure that aims to provide a standard protocol for data exchange between functions. It contains a batch (TensorDict

A group of Ray workers that can be managed collectively. This class extends WorkerGroup to provide Ray-specific functionality for creating an

A DataProto is a data structure that aims to provide a standard protocol for data exchange between functions. It contains a batch (TensorDict

Data structure for the output of reward calculations. Attributes: reward (float): The computed reward value based on the evaluation of th

Data structure for the output of reward calculations. Attributes: reward (float): The computed reward value based on the evaluation of th

A wrapper class for Ray actors with initialization arguments. This class extends ClassWithInitArgs to provide additional functionality for co

Manage FSDP checkpointing in SPMD training. - Saves/loads per-rank sharded model & optimizer states - Persists full lr_scheduler and RNG

Sharding manager to support data resharding when using FSDP + Ulysses

Used to count mfu during training loop Example: flops_counter = FlopsCounter(config) flops_achieved, flops_promised = flops_

Used to count mfu during training loop Example: flops_counter = FlopsCounter(config) flops_achieved, flops_promised = flops_

Define a resource pool specification. Resource pool will be initialized first.

A unified tracking interface for logging experiment data to multiple backends. This class provides a centralized way to log experiment metrics, p

A unified tracking interface for logging experiment data to multiple backends. This class provides a centralized way to log experiment metrics, p

Manage FSDP checkpointing in SPMD training. - Saves/loads per-rank sharded model & optimizer states - Persists full lr_scheduler and RNG

Sharding manager to support data resharding when using FSDP + Ulysses

Define a resource pool specification. Resource pool will be initialized first.

Load and preprocess RLHF data from Parquet files. - Caches files locally. - Reads into a HuggingFace Dataset and tokenizes prompts.

Sandbox to execute python code. WARNING: This class is for testing purpose only, do not use it in production. Please use a sandbox with stron

Note that this trainer runs on the driver process on a single CPU/GPU node.

Note that this trainer runs on the driver process on a single CPU/GPU node.

Data structure for input required to calculate rewards. Attributes: problem (str): The original problem text or prompt provided to the mo

Data structure for input required to calculate rewards. Attributes: problem (str): The original problem text or prompt provided to the mo

Reward function for evaluating mathematical answers. This class implements the __call__ method to process the input and determine the re

Reward function for evaluating mathematical answers. This class implements the __call__ method to process the input and determine the re

AsyncLLMServerManager manage a group of vllm instances, i.e AsyncvLLMServer.

DataProtoFuture aims to eliminate actual data fetching on driver. By doing so, the driver doesn't have to wait for data so that asynchronous

DataProtoFuture aims to eliminate actual data fetching on driver. By doing so, the driver doesn't have to wait for data so that asynchronous

The implementation of a simple digit completion task. The prompt is a sequence of numbers with fixed difference. The task is to complete the

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save -

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save -

A memory buffer is a contiguous torch tensor that may combine multiple tensors sharing with the underlying memory. It must have a unique type

A memory buffer is a contiguous torch tensor that may combine multiple tensors sharing with the underlying memory. It must have a unique type

Dataset for multi-turn conversations where each assistant response should be trained

The schema of parameters in OpenAI format.

The schema of a parameter in OpenAI format.

The schema of a function in OpenAI format.

The schema of a tool in OpenAI format.

Load and preprocess RLHF data from Parquet files. - Caches files locally. - Reads into a HuggingFace Dataset and tokenizes prompts.

This is an in-memory SFTDataset Arguments: config (OmegaConf): the data config

A test class to be imported by load_extern_type

Abstract base class for all tools that provides a common interface. All tools should inherit from this class and implement either: -

Abstract base class for all tools that provides a common interface. All tools should inherit from this class and implement either: -

Adaptive KL controller described in the paper: https://arxiv.org/pdf/1909.08593.pdf

Adaptive KL controller described in the paper: https://arxiv.org/pdf/1909.08593.pdf

Adaptive KL controller described in the paper: https://arxiv.org/pdf/1909.08593.pdf

Adaptive KL controller described in the paper: https://arxiv.org/pdf/1909.08593.pdf