A class that keeps all conversation history.

Subclass torch's LayerNorm to handle fp16 (by casting to float32 and back).

r""" Sampler that samples indices in a way that groups together features of the dataset of roughly the same length while keeping a bit of rand

A class that keeps all conversation history.

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LLaDADecoderLayer`] Args: config: LLaDAConfig

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LLaDADecoderLayer`] Args: config: LLaDAConfig

TaskManager indexes all tasks from the default `lmms_eval/tasks/` and an optional directory if provided.

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between

r""" This is the configuration class to store the configuration of a [`MplugOwlVisionModel`]. It is used to instantiate a mPLUG-Owl vision en

Fake file-like stream object that redirects writes to a logger instance.

r""" The ValueHead class implements a head for GPT2 that returns a scalar for each output token.

Adaptive KL controller described in the paper: https://arxiv.org/pdf/1909.08593.pdf

Construct the embeddings from word and position embeddings.

Iterable dataset that returns constant length chunks of tokens from stream of text files. The dataset also formats the text before tokenizati

Output class for the diffusers pipeline to be finetuned with the DDPO trainer Args: images (`torch.Tensor`): The generat

Output class for the diffusers scheduler to be finetuned with the DDPO trainer Args: latents (`torch.Tensor`): Predicted

Collate examples for DPO fine-tuning.

r""" DPO DataCollator class that pads the tokenized inputs to the maximum length of the batch. Args: pad_token_id (`int` defaults to 0

Dataset for DPODataset fine-tuning.

Collate examples for supervised fine-tuning.

Keeps track and saves relevant information of the evaluation process. Compiles the data from trackers and writes it to files, which can be pu

Hook class for implementing fast dLLM caching functionality in LLaDA model. This class handles both attention-level caching and generation-le

FilterEnsemble creates a pipeline applying multiple filters. Its intended usage is to stack multiple post-processing steps in order. `tas

Fixed KL controller.

Tracker for the evaluation parameters. Attributes: model_source (str): Source of the model (e.g. Hugging Face, GGUF, etc.) m

LLaDARotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla

LLaDARotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla

LLaDARotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev

LLaDARotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev

Multi-headed attention from 'Attention Is All You Need' paper

Transformer encoder consisting of `config.num_hidden_layers` self attention layers. Each layer is a [`MplugOwlVisionEncoderLayer`]. Args

A filter used to extract a model's answer on multiple choice questions with letter answers. assumes each document has a "choices" field c

Criteria to stop on the specified multi-token sequence.

r""" Class for tracking statistics per prompt. Mainly used to calculate advantage for the DPPO algorithm Args: buffer_size (`int`):