Modified from `transformers.models.llama.modeling_deepseek_v3.DeepseekV3Attention` add support for attention bias and softcapping

Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`DeepseekV2DecoderLayer`] Args: config: Deepse

Data collator used for padding free approach. Does the following: - concatate the entire mini batch into single long sequence [1, total_toke

DeepseekV2RotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla

DeepseekV2RotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev

A mixed expert module containing shared experts.

Multi-headed attention from 'Attention Is All You Need' paper

A mixed expert module containing shared experts.

GLM-4.7-Flash attention serving a GQLA checkpoint via MLA absorb. Same shapes as vanilla MLA — ``kv_b_proj`` is sized for ``num_heads`` and w

Swap in GQLA-absorb attention; reuse parent forward + MoE/MLP wiring.

Standalone twin of Glm4MoeLiteModel for the GQLA-absorb path. Cannot subclass Glm4MoeLiteModel directly (it is @support_torch_compile- decora

GLM-4.7-Flash attention with GQA-shaped KV. Requires q LoRA (always true on GLM-4.7).

Swap in GQLA attention; reuse parent forward + MoE/MLP wiring.

Standalone twin of Glm4MoeLiteModel. Cannot subclass it because the parent is @support_torch_compile-decorated; subclassing would break the c

Represents a single inference request for benchmarking.

The trick function of adding auxiliary (aux) loss, which includes the gradient of the aux loss during backpropagation.

Multi-headed attention from 'Attention Is All You Need' paper

r""" This is the configuration class to store the configuration of a [`DeepseekV2Model`]. It is used to instantiate an DeepSeek model accordin

DeepseekV2 flash attention module. This module inherits from `DeepseekV2Attention` as the weights of the module stays untouched. The only req

Main reference: DeepseekV2 paper, and FlashInfer Implementation (https://arxiv.org/abs/2405.04434 and https://github.com/flashinfer-ai/flashi

r""" This is the configuration class to store the configuration of a [`DeepseekV3Model`]. It is used to instantiate an DeepSeek model accordin

Top-level causal LM serving a GQLA checkpoint via MLA absorb.

Top-level causal LM. Same as parent except ``self.model``.