github.com/InternLM/InternLM-XComposer @main

<img src="https://github.com/InternLM/InternLM-XComposer/raw/main/assets/logo_en.png" width="650"/>






<b><font size="6">InternLM-XComposer-2.5</font></b>








    InternLM-XComposer2.5 <a href="https://huggingface.co/internlm/internlm-xcomposer2d5-7b">🤗</a> <a href="https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-xcomposer2d5-7b"><img src="https://github.com/InternLM/InternLM-XComposer/raw/main/assets/modelscope_logo.png" width="20px"></a> &nbsp｜ XComposer2.5 Technical Report <a href="https://arxiv.org/abs/2407.03320">  📄 </a>

English | 简体中文

Thanks the community for <a href="https://huggingface.co/spaces/Willow123/InternLM-XComposer">HuggingFace Demo </a>  | <a href="https://openxlab.org.cn/apps/detail/WillowBreeze/InternLM-XComposer">OpenXLab Demo</a> of InternLM-XComposer-2.5.







👋 join us on <a href="https://discord.gg/xa29JuW87d" target="_blank">Discord</a> and <a href="https://r.vansin.top/?r=internwx" target="_blank">WeChat</a>

🔥🔥🔥 InternLM-XComposer2.5-Reward

We release InternLM-XComposer2.5-Reward 🤗 (IXC-2.5-Reward, ACL 2025 Findings), a simple yet effective multi-modal reward model, including training code, evaluation scripts, and parts of the traininig data. Please refer to the project page for details.

🔥🔥🔥 InternLM-XComposer2.5-OmniLive

We release InternLM-XComposer2.5-OmniLive, a comprehensive multimodal system for long-term streaming video and audio interactions. Please refer to the project page for details.

Multimodal Projects of Our Team

InternLM-XComposer-2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model

InternLM-XComposer-2.5-OmniLive: A Specialized Generalist Multimodal System for Streaming Video and Audio Interactions

InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output

InternLM-XComposer2-: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD

InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Models

InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition

ShareGPT4Video: Improving Video Understanding and Generation with Better Captions

ShareGPT4V: Improving Large Multi-modal Models with Better Captions

MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMs

DualFocus: Integrating Macro and Micro Perspectives in Multi-modal Large Language Models

InternLM-XComposer-2.5 excels in various text-image comprehension and composition applications, achieving GPT-4V level capabilities with merely 7B LLM backend. IXC-2.5 is trained with 24K interleaved image-text contexts, it can seamlessly extend to 96K long contexts via RoPE extrapolation. This long-context capability allows IXC-2.5 to perform exceptionally well in tasks requiring extensive input and output contexts.

• Ultra-High Resolution Understanding: IXC-2.5 enhances the dynamic resolution solution proposed in IXC2-4KHD with a native 560 × 560 ViT vision encoder, supporting high-resolution images with any aspect ratio.

• Fine-Grained Video Understanding: IXC-2.5 treats videos as a ultra-high-resolution composite picture consisting of tens to hundreds of frames, allowing it to capture fine details through dense sampling and higher resolution for each frame.

• Multi-Turn Multi-Image Dialogue: IXC-2.5 supports free-form multi-turn multi-image dialogue, allowing it to naturally interact with humans in multi-round conversations.

• Webpage Crafting: IXC-2.5 can be readily applied to create webpages by composing source code (HTML, CSS, and JavaScript) following text-image instructions.

• Composing High-Quality Text-Image Articles: IXC-2.5 leverages specially designed Chain-of-Thought (CoT) and Direct Preference Optimization (DPO) techniques to significantly enhance the quality of its written content.

• Awesome performance: IXC-2.5 has been evaluated on 28 benchmarks, outperforming existing open-source state-of-the-art models on 16 benchmarks. It also surpasses or competes closely with GPT-4V and Gemini Pro on 16 key tasks.

  

Please refer to Technical Report for more details.

Demo Video

🔥 For the best experience, please keep the audio on while enjoying the video.

https://github.com/InternLM/InternLM-XComposer/assets/147793160/8206f07f-3166-461e-a631-9cbcdec6ae75

Youtube Video

Please refer to Chinese Demo for the demo of the Chinese version.

News and Updates

• 2024.12.12 🎉🎉🎉 InternLM-XComposer2.5-7B-Reward is publicly available.
• 2024.12.12 🎉🎉🎉 InternLM-XComposer2.5-OmniLive-7B is publicly available.
• 2024.07.15 🎉🎉🎉 ModelScope Swift supports InternLM-XComposer2.5-7B for finetuning and inference.
• 2024.07.15 🎉🎉🎉 LMDepoly supports InternLM-XComposer2.5-7B for 4 bit quantization and inference.
• 2024.07.15 🎉🎉🎉 InternLM-XComposer2.5-7B-4bit is publicly available.
• 2024.07.03 🎉🎉🎉 InternLM-XComposer2.5-7B is publicly available.
• 2024.07.01 🎉🎉🎉 ShareGPT4V is accepted by ECCV2024.
• 2024.04.22 🎉🎉🎉 The finetune code of InternLM-XComposer2-VL-7B-4KHD-7B are publicly available.
• 2024.04.09 🎉🎉🎉 InternLM-XComposer2-4KHD-7B and evaluation code are publicly available.
• 2024.04.09 🎉🎉🎉 InternLM-XComposer2-VL-1.8B is publicly available.

• 2024.02.22 🎉🎉🎉 We release DualFocus, a framework for integrating macro and micro perspectives within MLLMs to enhance vision-language task performance.

• 2024.02.06 🎉🎉🎉 InternLM-XComposer2-7B-4bit and InternLM-XComposer-VL2-7B-4bit are publicly available on Hugging Face and ModelScope.

• 2024.02.02 🎉🎉🎉 The finetune code of InternLM-XComposer2-VL-7B are publicly available.

• 2024.01.26 🎉🎉🎉 The evaluation code of InternLM-XComposer2-VL-7B are publicly available.
• 2024.01.26 🎉🎉🎉 InternLM-XComposer2-7B and InternLM-XComposer-VL2-7B are publicly available on Hugging Face and ModelScope.
• 2024.01.26 🎉🎉🎉 We release a technical report for more details of InternLM-XComposer2 series.
• 2023.11.22 🎉🎉🎉 We release the ShareGPT4V, a large-scale highly descriptive image-text dataset generated by GPT4-Vision and a superior large multimodal model, ShareGPT4V-7B.
• 2023.10.30 🎉🎉🎉 InternLM-XComposer-VL achieved the top 1 ranking in both Q-Bench and Tiny LVLM.
• 2023.10.19 🎉🎉🎉 Support for inference on multiple GPUs. Two 4090 GPUs are sufficient for deploying our demo.
• 2023.10.12 🎉🎉🎉 4-bit demo is supported, model files are available in Hugging Face and ModelScope.
• 2023.10.8 🎉🎉🎉 InternLM-XComposer-7B and InternLM-XComposer-VL-7B are publicly available on ModelScope.
• 2023.9.27 🎉🎉🎉 The evaluation code of InternLM-XComposer-VL-7B are publicly available.
• 2023.9.27 🎉🎉🎉 InternLM-XComposer-7B and InternLM-XComposer-VL-7B are publicly available on Hugging Face.
• 2023.9.27 🎉🎉🎉 We release a technical report for more details of our model series.

Model Zoo

Model	Usage	Transformers(HF)	ModelScope(HF)	Release Date
InternLM-XComposer-2.5	Video Understanding, Multi-image Multi-tune Dialog, 4K Resolution Understanding, Web Craft, Article creation, Benchmark	🤗internlm-xcomposer2.5	internlm-xcomposer2.5	2024-07-03
InternLM-XComposer2-4KHD	4K Resolution Understanding, Benchmark, VL-Chat	🤗internlm-xcomposer2-4khd-7b	internlm-xcomposer2-4khd-7b	2024-04-09
InternLM-XComposer2-VL-1.8B	Benchmark, VL-Chat	🤗internlm-xcomposer2-vl-1_8b	internlm-xcomposer2-vl-1_8b	2024-04-09
InternLM-XComposer2	Text-Image Composition	🤗internlm-xcomposer2-7b	internlm-xcomposer2-7b	2024-01-26
InternLM-XComposer2-VL	Benchmark, VL-Chat	🤗internlm-xcomposer2-vl-7b	internlm-xcomposer2-vl-7b	2024-01-26
InternLM-XComposer2-4bit	Text-Image Composition	🤗internlm-xcomposer2-7b-4bit	internlm-xcomposer2-7b-4bit	2024-02-06
InternLM-XComposer2-VL-4bit	Benchmark, VL-Chat	[🤗internlm-xcomposer2-vl-7b-4bit](h