Transform a prompt and optional multimodal content into a chat format. This method is used for chat models that expect a specific con

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

r""" labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language mo

Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (

r""" Args: labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing

r""" labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression

Optionally select a random LoRA request and return its associated tokenizer. This method is used when LoRA parameters are pr

Validate a sequence based on prompt and output lengths. Default pruning criteria are copied from the original `sample_hf_requests` and `

Load data from the dataset path into self.data. This method must be overridden by subclasses since the method to load data w

Oversamples the list of requests if its size is less than the desired number. Args: requests (List[SampleRequest

MMLU prompt without choices

Process a single image input and return a multimedia content dictionary. Supports three input types: 1. Dictionary with raw image bytes

This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he

Abstract method to generate sample requests from the dataset. Subclasses must override this method to implement dataset-specific log