A DataProto is a data structure that aims to provide a standard protocol for data exchange between functions. It contains a batch (TensorDict

DataProtoFuture aims to eliminate actual data fetching on driver. By doing so, the driver doesn't have to wait for data so that asynchronous

Class that holds a configuration for a generation task. A `generate` call supports the following generation methods for text-decoder, text-to

We assume the dataset contains a column that contains prompts and other information

Adaptive KL controller described in: https://arxiv.org/pdf/1909.08593.pdf Copied from https://github.com/huggingface/trl/blob/v0.11.0/trl/trainer

Class that holds arguments relative to `torch.compile` behavior, when using automatic compilation in `generate`. See [`torch.compile`](https:

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save -

Sharding manager to support data resharding when using FSDP + Ulysses

Fixed KL controller. Copeid from https://github.com/huggingface/trl/blob/v0.11.0/trl/trainer/utils.py#L72

Used to count mfu during training loop Example: flops_counter = FlopsCounter(config) flops_achieved, flops_promised = flops_

Note that this trainer runs on the driver process on a single CPU/GPU node.

Define a resource pool specification. Resource pool will be initialized first.

Using an enumeration class to avoid spelling errors in adv_estimator

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save -

Generic watermarking config

This class stores a class constructor and the args/kwargs to construct the class. It is used to instantiate the remote class.

Reward manager for rule-based reward.

Possible generation modes, downstream of the [`~generation.GenerationMixin.generate`] method.

Type of precision used.

The resource pool with meta info such as world size.

To create more roles dynamically, you can subclass Role and add new members

A runner for RL training.

Class that holds arguments for watermark generation and should be passed into `GenerationConfig` during `generate`. See [this paper](https://

Class that holds arguments for watermark generation and should be passed into `GenerationConfig` during `generate`. See [this paper](https://

A (distributed) worker.

A group of workers