<img src="https://model-demo.oss-cn-hangzhou.aliyuncs.com/Qwen3-VL-Embedding.png" width="400"/>
<img src="https://model-demo.oss-cn-hangzhou.aliyuncs.com/Qwen3-VL-Reranker.png" width="400"/>

Qwen3-VL-Embedding & Qwen3-VL-Reranker

State-of-the-art multimodal embedding and reranking models built on Qwen3-VL, supporting text, images, screenshots, videos, and mixed-modal inputs for advanced information retrieval and cross-modal understanding.

---

Table of Contents

• Overview
• Features
• Model Architecture
• Installation
• Usage
• Examples
• Model Performance
• Citation

---

Overview

The Qwen3-VL-Embedding and Qwen3-VL-Reranker model series are the latest additions to the Qwen family, built upon the recently open-sourced and powerful Qwen3-VL foundation model. Specifically designed for multimodal information retrieval and cross-modal understanding, this suite accepts diverse inputs including text, images, screenshots, and videos, as well as inputs containing a mixture of these modalities.

Building on the success of our text-oriented Qwen3-Embedding and Qwen3-Reranker series, these multimodal models extend best-in-class performance to visual and video understanding tasks. The models work in tandem: the Embedding model handles the initial recall stage by generating semantically rich vectors, while the Reranking model manages the re-ranking stage with precise relevance scoring, significantly enhancing final retrieval accuracy.

---

Features

• 🎨 Multimodal Versatility: Seamlessly process inputs containing text, images, screenshots, and video within a unified framework. Achieve state-of-the-art performance across diverse tasks including image-text retrieval, video-text matching, visual question answering (VQA), and multimodal content clustering.

• 🔄 Unified Representation Space: Leverage the Qwen3-VL architecture to generate semantically rich vectors that capture both visual and textual information in a shared space, facilitating efficient similarity estimation and retrieval across different modalities.

• 🎯 High-Precision Reranking: The reranking model accepts input pairs (Query, Document)—where both can consist of arbitrary single or mixed modalities—and outputs precise relevance scores for superior retrieval accuracy.

• 🌍 Exceptional Practicality:

• Support for over 30 languages, ideal for global applications
• Customizable instructions for task-specific optimization
• Flexible vector dimensions with Matryoshka Representation Learning (MRL)
• Strong performance with quantized embeddings for efficient deployment
• Easy integration into existing retrieval pipelines

Supported Languages

Qwen3-VL-Embedding and Qwen3-VL-Reranker supports 33 languages:

• English
• Chinese
• Spanish
• French
• Arabic
• Portuguese
• Russian
• Urdu
• Indonesian
• German
• Japanese
• Vietnamese
• Turkish
• Korean
• Persian
• Italian
• Thai
• Polish
• Ukrainian
• Uzbek
• Romanian
• Dutch
• Kazakh
• Greek
• Czech
• Swedish
• Serbian
• Hebrew
• Danish
• Finnish
• Norwegian
• Slovenian
• Irish

---

Model Architecture

Model Specifications

Model	Size	Layers	Sequence Length	Embedding Dimension	Quantization Support	MRL Support	Instruction Aware
Qwen3-VL-Embedding-2B	2B	28	32K	2048	✅	✅	✅
Qwen3-VL-Embedding-8B	8B	36	32K	4096	✅	✅	✅
Qwen3-VL-Reranker-2B	2B	28	32K	-	-	-	✅
Qwen3-VL-Reranker-8B	8B	36	32K	-	-	-	✅

LoRA Configs

Model	rank	alpha	target_modules
Qwen3-VL-Embedding	32	32	q_proj v_proj k_proj up_proj down_proj gate_proj
Qwen3-VL-Reranker	32	32	q_proj v_proj k_proj up_proj down_proj gate_proj

Architecture Design

Qwen3-VL-Embedding: Dual-Tower Architecture - Receives single-modal or mixed-modal input and maps it into a high-dimensional semantic vector - Extracts the hidden state vector corresponding to the [EOS] token from the base model's last layer as the final semantic representation - Enables efficient, independent encoding necessary for large-scale retrieval

Qwen3-VL-Reranker: Single-Tower Architecture - Receives an input pair (Query, Document) and performs pointwise reranking - Utilizes Cross-Attention mechanism for deeper, finer-grained inter-modal interaction and information fusion - Expresses relevance score by predicting the generation probability of special tokens (yes and no)

Feature Comparison

	Qwen3-VL-Embedding	Qwen3-VL-Reranker
Core Function	Semantic Representation, Embedding Generation	Relevance Scoring, Pointwise Re-ranking
Input	Single modality or mixed modalities	(Query, Document) pair with single- or mixed-modal inputs
Architecture	Dual-Tower	Single-Tower
Mechanism	Efficient Retrieval	Deep Inter-Modal Interaction, Precise Alignment
Output	Semantic Vector	Relevance Score

Both models are built through a multi-stage training paradigm that fully leverages the powerful general multimodal semantic understanding capabilities of Qwen3-VL, providing high-quality semantic representations and precise re-ranking mechanisms for complex, large-scale multimodal retrieval tasks.

---

Installation

Setup Environment

# Clone the repository
git clone https://github.com/QwenLM/Qwen3-VL-Embedding.git
cd Qwen3-VL-Embedding

# Run the script to setup the environment
bash scripts/setup_environment.sh

The setup script will automatically: - Install uv if not already installed - Install all project dependencies

After setup completes, activate the environment:

source .venv/bin/activate

Download Models

Our models are available on both Hugging Face and ModelScope.

Model	Hugging Face	ModelScope
Qwen3-VL-Embedding-2B	Link	Link
Qwen3-VL-Embedding-8B	Link	Link
Qwen3-VL-Reranker-2B	Link	Link
Qwen3-VL-Reranker-8B	Link	Link

Install download dependencies:

Download from Hugging Face:

uv pip install huggingface-hub

huggingface-cli download Qwen/Qwen3-VL-Embedding-2B --local-dir ./models/Qwen3-VL-Embedding-2B

Download from ModelScope:

uv pip install modelscope

modelscope download --model qwen/Qwen3-VL-Embedding-2B --local_dir ./models/Qwen3-VL-Embedding-2B

Usage

Quick Start

Embedding Model

Transformers usage

import torch
from src.models.qwen3_vl_embedding import Qwen3VLEmbedder

model = Qwen3VLEmbedder(
    model_name_or_path="./models/Qwen3-VL-Embedding-2B",
    # flash_attention_2 for better acceleration and memory saving
    # torch_dtype=torch.bfloat16, 
    # attn_implementation="flash_attention_2"
)

inputs = [{
    "text": "A woman playing with her dog on a beach at sunset.",
    "instruction": "Retrieve images or text relevant to the user's query.",
}, {
    "text": "A woman shares a joyful moment with her golden retriever on a sun-drenched beach at sunset, as the dog offers its paw in a heartwarming display of companionship and trust."
}, {
    "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"
}, {
    "text": "A woman shares a joyful moment with her golden retriever on a sun-drenched beach at sunset, as the dog offers its paw in a heartwarming display of companionship and trust.", 
    "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"
}]

embeddings = model.process(inputs)
print(embeddings @ embeddings.T)

vLLM usage

Note: Requires vLLM >= 0.14.0

For vLLM usage examples with the embedding model, please refer to examples/embedding_vllm.ipynb.

Reranking Model

Transformers usage

import torch
from src.models.qwen3_vl_reranker import Qwen3VLReranker

model = Qwen3VLReranker(
    model_name_or_path="./models/Qwen3-VL-Reranker-2B",
    # flash_attention_2 for better acceleration and memory saving
    # torch_dtype=torch.bfloat16, 
    # attn_implementation="flash_attention_2"
)

inputs = {
    "instruction": "Retrieve images or text relevant to the user's query.",
    "query": {"text": "A woman playing with her dog on a beach at sunset."},
    "documents": [
        {"text": "A woman shares a joyful moment with her golden retriever on a sun-drenched beach at sunset, as the dog offers its paw in a heartwarming display of companionship and trust."},
        {"image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"},
        {"text": "A woman shares a joyful moment with her golden retriever on a sun-drenched beach at sunset, as the dog offers its paw in a heartwarming display of companionship and trust.", 
         "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"}
    ],
    "fps": 1.0, 
    "max_frames": 64
}

scores = model.process(inputs)
print(scores)

vLLM usage

Note: Requires vLLM >= 0.14.0

For vLLM usage examples with the reranker model, please refer to examples/reranker_vllm.ipynb.

Model Input Specification

Multimodal Object

A dictionary that can contain the following keys: - text: Text input as a string or a list of strings - image: Image input, supports: - Local file path - URL (network path) - PIL.Image.Image instance - List of any combination of the above (multiple images) - video: Video input, supports: - Local file path - URL (network path) - Sequence of video frames (list of image paths or PIL.Image.Image instances) - List of any combination of the above (multiple videos)

Note: All input types (text, image, video) now support both single objects and lists of objects, allowing you to provide multiple inputs of the same type in a single request. For example, you can pass multiple images as a list, multiple text strings as a list, or multiple videos as a list.

Instruction

Task description for relevance evaluation (default: "Represent the user's input")

Video Sampling Settings

Only effective when video input is a video file: - fps: Frame sampling rate per second (frames per second) - max_frames: Maximum number of frames to sample

Input Format

Embedding Model: A list of dictionaries, where each dictionary contains: - Instruction (optional) - Video sampling settings (optional) - Multimodal object keys (text, image, and/or video)

Reranking Model: A dictionary containing: - query: A multimodal object - documents: A list of multimodal objects - instruction: Task description (optional) - fps: Video sampling rate (optional) - max_frames: Maximum frames (optional)

Embedding Model

Model Initialization Parameters

Qwen3VLEmbedder(
    model_name_or_path="./models/Qwen3-VL-Embedding-2B",
    max_length=8192,           # Default context length
    min_pixels=4096,           # Minimum pixels for input images
    max_pixels=1843200,        # Maximum pixels for input images (equivalent to 1280×1440 resolution)
    total_pixels=7864320,      # Maximum total pixels for input videos (multiplied by 2 in model)
                              # For a 16-frame video, each frame can have up to 983040 pixels (1280×768 resolution)
    fps=1.0,                   # Default sampling frame rate for video files (frames per second)
    max_frames=64,             # Maximum number of frames for video input
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2"
)

---

Examples

Embedding Model

We provide comprehensive examples here demonstrating various tasks across different modalities:

Text Tasks: - Text Classification (AG News) - Text Question Answering (SQuAD) - Text Retrieval (MS MARCO)

Image Tasks: - Image Classification (CIFAR-10) - Image Question Answering (VQAv2) - Image Retrieval (MS COCO)

Examples for video and visual document tasks are presented in the appendix of technical report

We also provide an end-to-end multimodal RAG exa