A class that keeps all conversation history.

A class that keeps all conversation history.

Track a series of values and provide access to smoothed values over a window or the global series average.

Track a series of values and provide access to smoothed values over a window or the global series average.

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`PhiDecoderLayer`] Args: config: PhiConfig

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`PhiDecoderLayer`] Args: config: PhiConfig

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between

Construct the embeddings from word and position embeddings.

Construct the embeddings from word and position embeddings.

Causal Language Modeling head. Reference: Improving Language Understanding by Generative Pre-Training. https://cdn.openai.com/res

Causal Language Modeling head. Reference: Improving Language Understanding by Generative Pre-Training. https://cdn.openai.com/res

Causal Language Modeling loss. Reference: Improving Language Understanding by Generative Pre-Training. https://cdn.openai.com/res

Causal Language Modeling loss. Reference: Improving Language Understanding by Generative Pre-Training. https://cdn.openai.com/res

r"""Dataset for chaining multiple :class:`DataPipeline` s. This class is useful to assemble different existing dataset streams. The chaining

r"""Dataset for chaining multiple :class:`DataPipeline` s. This class is useful to assemble different existing dataset streams. The chaining

This is a preliminary implementation to centralize and validate the configuration. May be altered in the future. A helper class to valid

This is a preliminary implementation to centralize and validate the configuration. May be altered in the future. A helper class to valid

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Token embedding with dropout.

Token embedding with dropout.

Inference parameters passed to model to efficiently calculate and store context during inference. Reference: https://github.com/Dao-A

Inference parameters passed to model to efficiently calculate and store context during inference. Reference: https://github.com/Dao-A

A wrapper to convert DataLoader as an infinite iterator. Modified from: https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/i

A wrapper to convert DataLoader as an infinite iterator. Modified from: https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/i

Subclass torch's LayerNorm to handle fp16.

Subclass torch's LayerNorm to handle fp16.

Multi-head attention layer.

Multi-head attention layer.

Multi-Layer Perceptron. Reference: Attention Is All You Need. https://arxiv.org/pdf/1706.03762.pdf.

Multi-Layer Perceptron. Reference: Attention Is All You Need. https://arxiv.org/pdf/1706.03762.pdf.

A utility class to create string representation of available model architectures and types. >>> from minigpt4.models import model_zoo >>

A utility class to create string representation of available model architectures and types. >>> from minigpt4.models import model_zoo >>

A simple wrapper for iterating over multiple iterators. Args: loaders (List[Loader]): List of Iterator loaders. ratios (List

A simple wrapper for iterating over multiple iterators. Args: loaders (List[Loader]): List of Iterator loaders. ratios (List