Initialize the RMSNorm normalization layer. Args: dim (int): The dimension of the input tensor. eps (float,

A fixture to create an instance of BitNetTransformer model

Forward pass through the RMSNorm layer. Args: x (torch.Tensor): The input tensor. Returns: torch.Te

torch.repeat_interleave(x, dim=2, repeats=n_rep)

Replaces all instances of nn.Linear in the given model with BitLinear15b. Args: model (nn.Module): The model to modify. Returns

Apply the RMSNorm normalization to the input tensor. Args: x (torch.Tensor): The input tensor. Returns:

Per token quantization to 8bits. No grouping is needed for quantization Args: x (Tensor): _description_ Returns: _type_: _de

Per token quantization to 8bits. No grouping is needed for quantization Args: x (Tensor): _description_ Returns: _type_: _de

Per token quantization to 8bits. No grouping is needed for quantization Args: x (Tensor): _description_ Returns: _type_: _de

Apply rotary embeddings to input tensors using the given frequency tensor. This function applies rotary embeddings to the given query 'xq' a

Flows the gradient from the output to the input. Returns two new tensors. Args: ctx : A_in : (B, L, D, N), X : (B, D, L,

Decodes a sequence of tokens into a string.

Generates autoregressive sequences based on the given start tokens. Args: start_tokens (torch.Tensor): The initial token

Returns the platform name as used in wheel filenames.

Precompute the frequency tensor for complex exponentials (cis) with given dimensions. This function calculates a frequency tensor with compl

Reshape frequency tensor for broadcasting it with another tensor. This function reshapes the frequency tensor to have the same shape as the

Scaled dot product attention with support for grouped queries. Einstein notation: - b: batch size - n / s: sequence length - h: numbe

Initialize the Attention module. Args: args (ModelArgs): Model configuration parameters. Attributes:

Initialize the FeedForward module. Args: dim (int): Input dimension. hidden_dim (int): Hidden dimension of t

Initialize a TransformerBlock. Args: layer_id (int): Identifier for the layer. args (ModelArgs): Model confi

Initialize a Transformer model. Args: params (ModelArgs): Model configuration parameters. Attributes:

Parameters ---------- device : str, optional The device to run the model on ('cpu' or 'cuda'). By default, 'cuda'

Decodes a token into a character.

Applies the parallel scan operation, as defined above. Returns a new tensor. Args: A_in : (B, L, D, N) X_in

Forward pass of the attention module. Args: x (torch.Tensor): Input tensor. start_pos (int): Starting positi

Perform a forward pass through the TransformerBlock. Args: x (torch.Tensor): Input tensor. start_pos (int):

Perform a forward pass through the Transformer model. Args: tokens (torch.Tensor): Input token indices. star

Forward pass of the BitLora layer. Args: x (Tensor): The input tensor. Returns: Tensor: The output

Forward pass of the BitLinear layer. Args: x (Tensor): The input tensor. Returns: Tensor: The outpu

Forward pass of the NoisyTopkRouter module. Args: mh_output (torch.Tensor): The output tensor from the multihead self at

Forward pass of the BitFeedForward module. Args: x (torch.Tensor): The input tensor. Returns: torch

Forward pass of the BitLinear layer. Args: x (Tensor): The input tensor. Returns: Tensor: The outpu