Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen2DecoderLayer`] Args: config: Qwen2Config

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Qwen3DecoderLayer`] Args: config: Qwen3Config

Multi-headed attention from 'Attention Is All You Need' paper

Multi-headed attention from 'Attention Is All You Need' paper

r""" The layer implementaion for [Parallelizing Linear Transformers with the Delta Rule over Sequence Length](https://arxiv.org/abs/2406.06484).

The layer implementaion for [Gated Delta Networks: Improving Mamba2 with Delta Rule](https://arxiv.org/abs/2412.06464). # noqa Similar to M

Compute ∆, A, B, C, and D the state space parameters and compute the `contextualized_states`. A, D are input independent (see Mamba paper [1]

r""" This is the configuration class to store the configuration of a [`Qwen2Model`]. It is used to instantiate a Qwen2 model according to the

r""" Args: _layer_implementation (`str`, *optional*, defaults to 'Qwen2DecoderLayer'): Decoder layer implementation. _

Construct a Qwen2 tokenizer. Based on byte-level Byte-Pair-Encoding. Same with GPT2Tokenizer, this tokenizer has been trained to treat space

Construct a "fast" Qwen2 tokenizer (backed by HuggingFace's *tokenizers* library). Based on byte-level Byte-Pair-Encoding. Same with GPT

r""" This is the configuration class to store the configuration of a [`Qwen3Model`]. It is used to instantiate a Qwen3 model according to the

r""" Args: _layer_implementation (`str`, *optional*, defaults to 'Qwen3DecoderLayer'): Decoder layer implementation. _