InternVL-U is the official open-source project developed by the InternVL-U team at Shanghai AI Laboratory. It is a 4B-parameter unified multimodal model (UMM) that aims to integrate multimodal understanding, reasoning, image generation, and image editing into a single framework, promoting the widespread adoption of all-around multimodal intelligence at an efficient and practical model scale.
Use Cases The model targets scenarios that require unified handling of multiple multimodal tasks: it can perform multimodal understanding and reasoning as well as image generation and image editing. Officially, it aims to serve as a strong baseline to accelerate all-around UMM research progress toward AGI.
Key Capabilities
- Unified yet modular design: based on unified context modeling, with modality-specific modular structures and decoupled visual representations.
- Strong backbone + strong generator: combines an advanced multimodal large language model (MLLM) with a dedicated MMDiT-based visual generation head.
- High-quality data synthesis: builds a complete data pipeline for high-semantic-density tasks (e.g., text rendering and editing, scientific reasoning), using chain-of-thought (CoT) to align abstract intent with precise visual execution.
- Performance and efficiency: at a limited parameter scale, officially it outperforms unified open-source UMM baselines on generation and editing while retaining strong multimodal understanding and reasoning capabilities.
- Multi-image understanding and reasoning: since 2026/03/19, multi-image understanding and reasoning is supported, and the examples in Quick Start can be used.
Usage Instructions for installing dependencies are provided in the official repository; before use, all required dependencies must be installed. Model checkpoints are released on Hugging Face. The repository provides a demo to showcase InternVL-U's unified pipeline for multimodal understanding, image generation, and image editing, with optional output of reasoning guidance (text + image) results. Regarding generation modes, the official guidance notes: when performing image generation or editing, if generation_mode is set to "image", the CoT text reasoning mode will not be activated; officially, this mode is recommended in most scenarios, especially simple scenarios and instructions, because it can already produce good results. If the user input involves more challenging reasoning, it is recommended to set generation_mode to "text_image" to enable more complex generation.
Release Notes 2026/03/06: Released the InternVL-U technical report (arXiv); released inference code and model checkpoints (Hugging Face); released GenEditEvalKit—a unifiedEvaluation toolkit that helps developers efficiently manage large-scale benchmark inference and evaluation for UMM as well as image generation and editing models; releases TextEdit Benchmark—a high-quality multi-scenario benchmark for evaluating the text editing capabilities of image generation models. 2026/03/19: Support multi-image understanding and reasoning.
Applicable scenarios Suitable for multimodal understanding and reasoning, image generation, and image editing, especially high-semantic-density tasks involving text rendering and editing, scientific reasoning, etc., as well as research and development work that requires a unified framework covering understanding, reasoning, generation, and editing.
Limitations and missing information Official materials do not provide specific facts such as pricing, charging models, license agreement details, supported platforms, hardware requirements, or performance figures; this content is not yet available. Before use, please refer to the actual instructions on the repository and Hugging Face pages.
