Encode text (question/keywords) into text token embeddings for coverage. Uses LLaVA embed_tokens; batch size must be 1. No BOS.

Apply coverage-based subset selection: select vision tokens that cover text tokens (question) and the vision token set (multimodal co

Filter stopwords and return descriptive text for text token embedding (coverage).

Extract question text from Qwen2.5-VL message format. User messages may have content as str or list of {"type": "text", "text": "..."} / {"ty

Inject MMTok into a LLaVA model: coverage-based subset selection of vision tokens. A subset of vision tokens is selected under the maximum c

Inject MMTok token selection into Qwen2.5-VL. Qwen2.5-VL uses dynamic resolution, so the number of vision tokens per image varies. MMTok

Subset selection under maximum coverage: greedy selection of vision tokens to cover text tokens (question) and the vision token set.

Initialize token selector and text processor.

Apply monkey-patch to llava.mm_utils.process_images so that latest images and padding patch indices are stored for MMTok. Idempotent; safe to

Coverage-based subset selection for Qwen2.5-VL: select vision tokens to cover text (question only, or question+answer if provided) an

Compute patch indices that fall inside the padding region (after resize to target_size). Args: original_size: (width, height) of the

Encodes images into continuous embeddings that can be forwarded to the language model. Args: pixel_values (`torch.FloatT

Return padding patch indices (List[List[int]]) set by the last process_images call.

Encodes videos into continuous embeddings that can be forwarded to the language model. Args: pixel_values_videos (`torch

Greedy maximum-coverage selection (JIT). Combined = [P; Q*alpha]: P = text–vision coverage, Q = vision–vision coverage (diversity). Args

Greedy maximum-coverage subset selection. Combined = [P; Q*alpha]: cover text tokens (P) and vision tokens (Q). Uses JIT kern

Patch the 'copy' method of a conversation template to capture the prompt for MMTok. Imports llava.conversation only inside this function so L

Patch processor.apply_chat_template to capture question text and set it on mmtok_model_instance.

Enable/disable padding index computation in the process_images wrapper (LLaVA-1.5 only).

Get question from qwen_model.model.

Set question on qwen_model; stored on model.

Forward with coverage-based subset selection: CLIP -> mm_projector -> select vision tokens that cover text (question) and vision set;

Args: hidden_states (`torch.Tensor` of shape `(seq_len, hidden_size)`): The final hidden states of the model.

r""" pixel_values_videos (`torch.FloatTensor` of shape `(seq_length, num_channels * temporal_size * image_size * image_size)): The

Patch processor.apply_chat_template on the lmms_eval Qwen wrapper to capture question.