A class that keeps all conversation history.

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Phi3DecoderLayer`] Args: config: Phi3Config

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`PhiDecoderLayer`] Args: config: PhiConfig

Fake file-like stream object that redirects writes to a logger instance.

Collate examples for supervised fine-tuning.

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Vision Transformer with support for patch or hybrid CNN input stage

Dataset for supervised fine-tuning.

r""" Sampler that samples indices in a way that groups together features of the dataset of roughly the same length while keeping a bit of rand

https://arxiv.org/abs/2212.00794

Image to Patch Embedding

PhiRotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla

PhiRotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev

Subclass torch's LayerNorm (with cast back to input dtype).

Multi-headed attention from 'Attention Is All You Need' paper

r""" This is the configuration class to store the configuration of a [`Phi3Model`]. It is used to instantiate a Phi-3 model according to the s

Phi-3 flash attention module. This module inherits from `Phi3Attention` as the weights of the module stays untouched. The only required chang

Phi3 attention module using torch.nn.functional.scaled_dot_product_attention. This module inherits from `Phi3Attention` as the weights of the

Multi-headed attention from 'Attention Is All You Need' paper

r""" This is the configuration class to store the configuration of a [`PhiModel`]. It is used to instantiate an Phi model according to the spe

Phi flash attention module. This module inherits from `PhiAttention` as the weights of the module stays untouched. The only required change w

Different separator style.