Qwen2-VL is a series of multimodal large language models developed by the Qwen team at Alibaba Cloud. This GitHub repository page primarily introduces its subsequent iteration, Qwen3-VL, describing it as the most powerful vision-language model in the Qwen series to date. This generation has undergone comprehensive upgrades in text understanding and generation, visual perception and reasoning, context length, spatial and video dynamic understanding, and agent interaction capabilities.
Purpose and Positioning Qwen3-VL is aimed at multimodal understanding and generation tasks, and can be used for visual agents operating PC/mobile GUIs (recognizing elements, understanding functions, invoking tools, completing tasks), visual coding (generating Draw.io/HTML/CSS/JS from images/videos), advanced spatial perception (judging object position, viewpoint, and occlusion, providing stronger 2D grounding and supporting 3D grounding for spatial reasoning and embodied AI), long context and video understanding (native 256K context, expandable to 1M, supporting processing of books and hours-long videos, with full recall and second-level indexing), enhanced multimodal reasoning (STEM/math, causal analysis, and answers based on logic and evidence), upgraded visual recognition (can recognize celebrities, anime, products, landmarks, animals, plants, etc.), expanded OCR (supports 32 languages, an increase from the previous 10, more robust in low light, blur, and tilt conditions, better handling of rare/ancient scripts and terminology, and improved long document structure parsing), and text understanding (comparable to pure LLMs, achieving lossless unified text-visual understanding).
Main Capabilities The model offers two architectures, Dense and MoE, covering deployment needs from edge to cloud, and has Instruct and reasoning-enhanced Thinking versions. Technically, it includes Interleaved-MRoPE (allocating full frequency across time, width, and height through robust positional embeddings to enhance long-duration video reasoning), DeepStack (fusing multi-level ViT features to capture fine-grained details and sharpen image-text alignment), and text-timestamp alignment (going beyond T-RoPE to achieve precise, timestamp-anchored event localization and enhance video temporal modeling).
Version Releases 2024.08.30 Released the Qwen2-VL series, with 2B and 7B models available, and the 72B open-source model coming soon. 2024.09.19 Released the instruction-tuned Qwen2-VL-72B and its quantized versions (AWQ, GPTQ-Int4, GPTQ-Int8), and also released the Qwen2-VL paper. 2024.12.25 Released QvQ-72B-Preview, an experimental research model focused on enhancing visual reasoning. 2025.01.28 Released the Qwen2.5-VL series. 2025.02.20 Released the Qwen2.5-VL technical report and provided AWQ quantized models for 3B, 7B, and 72B. 2025.03.25 Released Qwen2.5-VL-32B, which is smarter and whose responses align more with human preferences. 2025.04.08 Provided fine-tuning code for Qwen2-VL and Qwen2.5-VL. 2025.09.23 Released Qwen3-VL-235B-A22B-Instruct and Thinking versions. 2025.10.4 Released Qwen3-VL-30B-A3B-Instruct and Thinking, as well as FP8 versions. 2025.10.15 Released Qwen3-VL-4B and 8B (Instruct/Thinking). 2025.10.21 Released Qwen3-VL-2B and 32B (Instruct/Thinking). 2025.11.27 Released the Qwen3-VL paper.
Usage The page provides chat model code snippets based on 🤗 Transformers and ModelScope, and recommends that users in mainland China use ModelScope. Through the official HF processor, the visual token budget can be controlled: in the image processor, size['longest_edge'] corresponds to max_pixels (the maximum number of pixels in the image), and size['shortest_edge'] corresponds to min_pixels; in the video processor, size['longest_edge'] represents the total maximum number of pixels for all frames (T×H×W does not exceed this value), and size['shortest_edge'] sets the minimum total pixel budget for the video. The sampling fps or number of frames can also be controlled. The new version of qwen-vl-utils supports controlling pixel constraints by visual input, and adds "image_patch_size" (14 for Qwen2.5-VL, 16 for Qwen3-VL, default 14) and "return_video_metadata" (only for Qwen3-VL; if True, the video returns (video_tensor, video_metadata), default False). Note that qwen-vl-utils has already resized the image/video, so pass do_resize=False to the processor to avoid resizing again.
Input supports local files, base64, and URL images; you can specify exact dimensions (resized_height, resized_width, rounded to the nearest multiple of 32, 32 for Qwen3VL, 28 for Qwen2.5VL) or define min_pixels and max_pixels to scale while preserving aspect ratio. Video supports image lists, local paths, and URLs; it is recommended to set min_pixels and max_pixels to limit frame resolution, or use total_pixels to limit the total number of video tokens (recommended to be below 24576×32×32). qwen-vl-utils supports three video decoding backends: torchvision, decord, torchcodec; torchcodec is recommended (decord has issues such as decoding hangs and is no longer maintained). Non-Linux environments may be unable to install decord from PyPI; you can pip install qwen-vl-utils to fall back to torchvision; torchcodec must be installed manually and depends on FFmpeg. You can switch backends via FORCE_QWENVL_VIDEO_READER.
You can install the latest Flash Attention 2 and set attn_implementation="flash_attention_2" when loading the model (requires compatible hardware, and the model must be loaded with torch.float16 or torch.bfloat16). The current config.json context length is 256K tokens; when exceeded, YaRN extension can be used by modifying max_position_embeddings and rope_scaling; when using vLLM, you can add --rope-scaling and --max-model-len. Because Interleaved-MRoPE position IDs grow more slowly, a smaller scaling factor should be used, for example, when using a 256K length to support a 1M context, set factor=2 or 3 instead of 4.
Deployment and Inference You can use vLLM (requires vllm>=0.11.0 to support Qwen3-VL) or SGLang for efficient serving, and access it through an OpenAI-style API; the page provides examples for image requests, video requests, and offline inference. The startup command is suitable for H100/H200; for other GPUs, please refer to the vLLM community guide. It also provides steps for setting up a web-based UI demo, a prebuilt Docker image, and a Docker image qwenllm/qwenvl with a prebuilt environment; you only need to install drivers and download the model files to start the demo.
Evaluation and Fine-Tuning Evaluation reproduction uses vLLM inference; the evaluation frameworks are VLMEvalKit and lmms-eval; the prompts for some benchmarks have been fine-tuned, and a few benchmarks are internally built. The page mentions that fine-tuning code for Qwen2-VL and Qwen2.5-VL is provided.
Applicable Scenarios and Limitations Suitable for scenarios such as multimodal dialogue, document parsing, video understanding, visual agents, OCR, and spatial reasoning. In terms of limitations: Flash Attention 2 requires compatible hardware and specific data types; the decord backend has known issues and is no longer maintained; torchcodec depends on FFmpeg; long-context extension requires adjusting the scaling factor; the code and reproduction materials for some evaluation details and internal benchmarks are planned to be released later. Specific performance data, pricing standards, platform support, and other information are not yet provided.
