A class that keeps all conversation history.

Processes an answer similar to Eval AI copied from https://github.com/facebookresearch/mmf/blob/c46b3b3391275b4181567db80943473a8

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Phi3DecoderLayer`] Args: config: Phi3Config

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`PhiDecoderLayer`] Args: config: PhiConfig

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] Args: config: Qwen2Config

Fake file-like stream object that redirects writes to a logger instance.

Collate examples for supervised fine-tuning.

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Vision Transformer with support for patch or hybrid CNN input stage

Dataset for supervised fine-tuning.

r""" Sampler that samples indices in a way that groups together features of the dataset of roughly the same length while keeping a bit of rand

LlamaRotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla

LlamaRotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev

https://arxiv.org/abs/2212.00794

Image to Patch Embedding

PhiRotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla

PhiRotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev

Multi-headed attention from 'Attention Is All You Need' paper

Transformer encoder consisting of `config.num_hidden_layers` self attention layers. Each layer is a [`SigLipEncoderLayer`]. Args:

Multihead Attention Pooling.

A utility attention mask class that allows one to: - Create a causal 4d mask - Create a causal 4d mask with slided window

Subclass torch's LayerNorm (with cast back to input dtype).

Multi-headed attention from 'Attention Is All You Need' paper

r""" This is the configuration class to store the configuration of a [`LlamaModel`]. It is used to instantiate an LLaMA model according to the

Llama flash attention module. This module inherits from `LlamaAttention` as the weights of the module stays untouched. The only required chan

Llama attention module using torch.nn.functional.scaled_dot_product_attention. This module inherits from `LlamaAttention` as the weights of t

Construct a Llama tokenizer. Based on byte-level Byte-Pair-Encoding. The default padding token is unset as there is no padding token in the o