A class that keeps all conversation history.

A DataProto is a data structure that aims to provide a standard protocol for data exchange between functions. It contains a batch (TensorDict

r""" Sampler that samples indices in a way that groups together features of the dataset of roughly the same length while keeping a bit of rand

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between

DataProtoFuture aims to eliminate actual data fetching on driver. By doing so, the driver doesn't have to wait for data so that asynchronous

https://arxiv.org/abs/2212.00794

We assume the dataset contains a column that contains prompts and other information

Fake file-like stream object that redirects writes to a logger instance.

r""" The ValueHead class implements a head for GPT2 that returns a scalar for each output token.

Adaptive KL controller described in the paper: https://arxiv.org/pdf/1909.08593.pdf

Adaptive KL controller described in: https://arxiv.org/pdf/1909.08593.pdf Copied from https://github.com/huggingface/trl/blob/v0.11.0/trl/trainer

Construct the embeddings from word and position embeddings.

Iterable dataset that returns constant length chunks of tokens from stream of text files. The dataset also formats the text before tokenizati

Output class for the diffusers pipeline to be finetuned with the DDPO trainer Args: images (`torch.Tensor`): The generat

Output class for the diffusers scheduler to be finetuned with the DDPO trainer Args: latents (`torch.Tensor`): Predicted

Collate examples for DPO fine-tuning.

r""" DPO DataCollator class that pads the tokenized inputs to the maximum length of the batch. Args: pad_token_id (`int` defaults to 0

Dataset for DPODataset fine-tuning.

Collate examples for supervised fine-tuning.

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Vision Transformer with support for patch or hybrid CNN input stage

Vision Transformer with support for patch or hybrid CNN input stage

A checkpoint manager that saves and loads - model - optimizer - lr_scheduler - extra_states in a SPMD way. We save -

Sharding manager to support data resharding when using FSDP + Ulysses

Fixed KL controller.

Fixed KL controller. Copeid from https://github.com/huggingface/trl/blob/v0.11.0/trl/trainer/utils.py#L72

Used to count mfu during training loop Example: flops_counter = FlopsCounter(config) flops_achieved, flops_promised = flops_

HuggingFace model adapter

HuggingFace tokenizer wrapper

LlamaRotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla

LlamaRotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev

Transformer encoder consisting of `config.num_hidden_layers` self attention layers. Each layer is a [`MLCDEncoderLayer`]. Args: