A wrapper class for Ray actors with initialization arguments. This class extends ClassWithInitArgs to provide additional functionality for co

A group of Ray workers that can be managed collectively. This class extends WorkerGroup to provide Ray-specific functionality for creating an

The response from a tool execution.

A DataProto is a data structure that aims to provide a standard protocol for data exchange between functions. It contains a batch (TensorDict

HTTP-based adapter for SGLang engines. This adapter allows interaction with SGLang engines through HTTP requests instead of direct engine cal

A dispatcher that delegates to specific profilers based on config.tool. Supported tools: - nsys: NsightSystemsProfiler - npu: NPUProfiler

A metric aggregator for collecting and aggregating numeric values. This class accumulates numeric values (int, float, or scalar tensors) and

Worker profiler config. The inheritance from BaseConfig provides omegaconf.DictConfig-like interface for a dataclass config. Args: d

Async token bucket for rate limiting with variable token consumption. The token bucket algorithm is a classic rate limiting technique that allows

Configuration for critic model training. The inheritance from BaseConfig provides omegaconf.DictConfig-like interface for a dataclass config.

FSDP optimizer configuration extending base OptimizerConfig. Args: optimizer (str): Optimizer class name (e.g., "AdamW", "AdamW8bit", "_A

Reward manager with rate limiting for API-based reward functions. This manager implements a sophisticated three-layer rate limiting system de

Base optimizer configuration. Args: lr (float): learning rate. Must be specified. lr_warmup_steps_ratio (float): Warmup steps rat

NPU profiler too; config.

A unified tracking interface for logging experiment data to multiple backends. This class provides a centralized way to log experiment metrics, p

Agent loop manager that manages a group of agent loop workers.

Manage FSDP checkpointing in SPMD training. - Saves/loads per-rank sharded model & optimizer states - Persists full lr_scheduler and RNG

Sharding manager to support data resharding when using FSDP + Ulysses

Load and preprocess RLHF data from Parquet files. - Caches files locally. - Reads into a HuggingFace Dataset and tokenizes prompts.

Checkpoint engine manager to coordinate weight synchronization between trainer and rollout replicas. - ME: model engine, FSDP, MCore, VeOmni, exp

Configuration for FSDP (Fully Sharded Data Parallel). The inheritance from BaseConfig provides omegaconf.DictConfig-like interface for a dataclas

Define a resource pool specification. Resource pool will be initialized first.

Configuration for actor model training. The inheritance from BaseConfig provides omegaconf.DictConfig-like interface for a dataclass config.

Asynchronous HTTP-based adapter for SGLang engines. This class inherits from HttpServerAdapter and adds async capabilities for non-blocking H

Configuration for FSDP-based critic model training. The inheritance from CriticConfig provides all base critic configuration plus FSDP-specific s

Used to count mfu during training loop Example: flops_counter = FlopsCounter(config) flops_achieved, flops_promised = flops_

Configuration for Megatron parallelism. The inheritance from BaseConfig provides omegaconf.DictConfig-like interface for a dataclass config.

RolloutSkip skips sequence generation during rollout by attempting to load previously dumped data. If no dumped data is found, it generates n

Agent loop output.

Configuration for Mcore module wrapper.

Dataset for multi-turn conversations where each assistant response should be trained Args: data_files (str or list): Path(s) to Parq

The schema of a tool in OpenAI format.

Async broadcast operation with NCCL in separate thread. Args: rank (int): The rank of the current process. group_name (str): The

Async broadcast operation with HCCL in separate thread. Args: rank (int): The rank of the current process. group_name (str): The

FSDP DataParallel PPO Actor or Ref worker Args: config (ActorConfig): Actor config actor_module (nn.Module): Actor or ref module

评测配置（用于保存历史配置）

Mcore optimizer configuration extending base OptimizerConfig. Args: optimizer (str): Optimizer name; default is "adam". lr (float

Encapsulates a readable operation to remote agent. 1. send metadata to remote agent 2. wait until remote agent read complete. Args:

RewardLoopManager run in single controller. This class will create reward loop workers and manage them. RewardLoopManager will deprecate

A distributed worker that handles initialization and configuration for distributed training. This class manages worker initialization, configurat

Encapsulates all state variables for the agent loop. AgentData is passed to tool calling in case that tool may need to access full history state.

Configuration for the algorithm. The inheritance from BaseConfig provides omegaconf.DictConfig-like interface for a dataclass config. Args:

Configuration for checkpoint engine to update weights from trainer to rollout

A demo interaction for calculating the reward of gsm8k. - `start_interaction`: start a interaction instance for a trajectory. - `generate_res

2D 势能面环境管理器。 定义域: x, y in [0, 10] nm

Checkpoint manager for Megatron-LM distributed training. This class manages the saving and loading of model checkpoints in a Megatron-LM

Reward model manager.

This is an in-memory SFTDataset Arguments: config (OmegaConf): the data config

A custom collate_fn that handles batching of sequences. 1. for variable-length sequences, convert them into NestedTensors. 2. for fixed-l

管理PyBullet实例。每个Case对应一个独立的Context实例。

任务信息

TrainingWorker provides a Tinker-like API (https://thinkingmachines.ai/tinker/) as a RayWorkerGroup to a single controller. Currently, we onl

This worker can be instantiated as a standalone actor or a standalone rollout or a standalone reference policy or a hybrid engine based on th

A CAD modeling sequence, a series of extrude operations.

Checkpoint handler handles the path, global_step of a checkpoint folder. Currently, it only works with a single model. We can expand it t

Local coordinate system for sketch plane.

DataProtoFuture aims to eliminate actual data fetching on driver. By doing so, the driver doesn't have to wait for data so that asynchronous

Wrapper for DictConfig to avoid hydra.utils.instantiate recursive resolve.

分布式Master服务器，支持动态Worker注册和工具发现

基于胜率阈值的通用扑克 Bot。 可以通过调整阈值参数来模拟 '激进派' 或 '保守派'。

Single extrude operation with corresponding a sketch profile. NOTE: only support single sketch profile. Extrusion with multiple profiles is decomp

Configuration for FSDP actor models. The inheritance from BaseConfig provides omegaconf.DictConfig-like interface for a dataclass config. Ar