A container class used to store the whole dataset. Inside the class, each data sample will be stored in `Item` class. The properties of the datase

Base class for Actor models in reinforcement learning. This class serves as a foundation for implementing various actor models, which are re

A container class used to store and manipulate a sample within a dataset. Information related to this sample during training/inference will be sto

r"""Dense retriever based on pre-built faiss index.

Dataset for SFT model Args: dataset: dataset for SFT model tokenizer: tokenizer for SFT model max_length: max length

r""" Dataset attributes.

r""" A callback for saving the processor.

Encoder class for encoding queries using a specified model. Attributes: model_name (str): The name of the model. model_path

Samples is a batch of data. There can be 2 formats to store the samples, batched or packed. The batched format means padding is applied to the

Samples is a batch of data. There can be 2 formats to store the samples, batched or packed. The batched format means padding is applied to the

Samples is a batch of data. There can be 2 formats to store the samples, batched or packed. The batched format means padding is applied to the

Samples is a batch of data. There can be 2 formats to store the samples, batched or packed. The batched format means padding is applied to the

A group of ray actors Functions start with 'async' should return list of object refs Args: num_nodes (int): Number of nodes for

Dataset for PPO model Args: dataset: dataset for PPO model tokenizer: tokenizer for PPO model max_length: max length

Dataset for reward model Args: dataset: dataset for reward model self.tokenizer: self.tokenizer for reward model sel

r"""BM25 retriever based on pre-built pyserini index.

r""" General class for chat models. Backed by huggingface or vllm engines. Supports both sync and async methods. Sync methods: chat(), st

Evaluator is used to summarize the results of all metrics.

Experience is a batch of data. These data should have the the sequence length and number of actions. Left padding for sequences is applied.

r""" Arguments pertaining to which techniques we are going to fine-tuning with.

GPT Language Model Loss

A stopping criteria that halts the text generation process if any specified stop word is encountered. Inspired by https://discuss.huggingfac

r""" Computes and stores the average and current value.

ClipEncoder class for encoding queries using CLIP.

r""" A dummy optimizer used for the GaLore or APOLLO algorithm.

Experience is a batch of data. These data should have the the sequence length and number of actions. Left padding for sequences is applied.

Experience is a batch of data. These data should have the the sequence length and number of actions. Left padding for sequences is applied.

Experience is a batch of data. These data should have the the sequence length and number of actions. Left padding for sequences is applied.

r""" Data collator for pairwise data.

Dataset for process reward model Args: dataset: dataset for reward model self.tokenizer: self.tokenizer for reward model

Merge logits of different docs in one batch. Reference: fastRAG

STEncoder class for encoding queries using SentenceTransformers. Attributes: model_name (str): The name of the model. model_

Unpaired preference dataset for algorithm, like KTO Args: dataset: raw dataset self.tokenizer: self.tokenizer for model

Adaptive KL controller described in the paper: https://arxiv.org/pdf/1909.08593.pdf

BufferItem is an item of experience data. Shapes of each tensor: sequences: (S) action_log_probs: (A) values: (1) returns: (1)

Wrapper replacing None outputs by empty tensors, which allows the use of checkpointing.

r""" Computes reward accuracy and supports `batch_eval_metrics`.

r""" Computes accuracy and supports `batch_eval_metrics`.

r""" Computes text similarity scores and supports `batch_eval_metrics`. Wraps the tokenizer into metric functions, used in CustomSeq2SeqTrain

r""" Inherits PPOTrainer.

r""" Inherits Seq2SeqTrainer to compute generative metrics such as BLEU and ROUGE.

r""" Inherits Trainer for custom optimizer.

DPO Loss

Trainer for Direct Preference Optimization (DPO) training. Args: model (torch.nn.Module): The primary model to be trained. r

The strategy for training with Accelerator.

r""" Default tool using template.

r"""Sampler that restricts data loading to a subset of the dataset. It is especially useful in conjunction with :class:`torch.nn.parallel.Dis

Fixed KL controller.

r""" GLM-4 tool using template.

Creates a HF model that inherits from REPLUG_Generation class

r"""A tool class used to build an index used in retrieval.

Language Model Knowledge Distillation Loss

Trainer for Knowledge Distillation. Args: model (torch.nn.Module): The model to be trained. strategy (Strategy): The trainin

r""" Data collator for KTO data.

KTO loss for uneven sampling

Trainer for KTO training. Args: model (torch.nn.Module): The primary model to be trained. ref_model (torch.nn.Module): The r

Implementation for (Long)LLMLingua.