Kimi-VL - Moonshot AI's Open-Source Lightweight Multimodal Vision-Language Model

What is Kimi-VL

Kimi-VL is an open-source lightweight multimodal vision-language model from Moonshot AI, based on the lightweight MoE model Moonlight (16B total parameters, 2.8B activated parameters) and the native-resolution MoonViT vision encoder (400M parameters). Kimi-VL supports multimodal inputs such as single images, multiple images, videos, and long documents, and performs excellently in tasks including image perception, mathematics, multidisciplinary questions, and OCR, with particular advantages in long context (128K) and complex reasoning. It performs excellently in tasks such as mathematical reasoning and long video understanding, surpassingGPT-4oand other models. Kimi-VL has released a model version that supports long thinking, Kimi-VL-Thinking, based on long chain-of-thought fine-tuning and reinforcement learning. With only 2.8B activated parameters, on benchmarks with relatively high reasoning difficulty, some results approach or even exceed those of ultra-large frontier models.

The latest open-source Kimi-VL-A3B-Thinking-2506 version performs excellently on multimodal reasoning benchmarks, with greatly improved accuracy and a 20% reduction in thinking length. The model's visual understanding capability is significantly enhanced, supporting higher-resolution image processing, and has made notable progress on high-resolution perception and OS-agent grounding benchmarks (scoring 83.2 on V*Benchmark, 52.8 on ScreenSpot-Pro, and 52.5 on OSWorld-G). The model performs excellently in multiple fields such as image understanding, chart reasoning, and mathematical calculation, and supports specific response modes and chains of thought.

Kimi-VL's Main Features

• Multimodal information input: Supports multiple input forms such as single images, multiple images, videos, and long documents.
• Fine-grained image perception: Performs detailed analysis of images, identifying complex details and scenes in images.
• Mathematical and logical reasoning: Performs excellently in multimodal mathematical problems and logical reasoning tasks, supporting complex calculations combined with visual information.
• OCR and text recognition: Performs excellently in optical character recognition (OCR) tasks, accurately recognizing text content in images.
• Agent applications: Supports Agent tasks such as screen snapshot parsing and agent navigation, handling complex visual and textual interaction scenarios.
• Long thinking capability: The Kimi-VL-Thinking version supports deeper reasoning in complex tasks.

Kimi-VL's Technical Principles

•  Architecture Design：
  • Vision Encoder (MoonViT): Based on a 400M-parameter Vision Transformer architecture, processes images at native resolution without the need to split or stitch images. Introduces the packing method from NaViT, dividing images into image patches, flattening them, and concatenating them into a one-dimensional sequence, sharing the same operators and optimization methods as the language model.
  • Multilayer Perceptron Projection Module (MLP Projector): Bridges the vision encoder and language model based on a two-layer MLP. The pixel rearrangement operation compresses the spatial dimensions of image features and projects them to the embedding dimension required by the language model.
  • Mixture-of-Experts (MoE) Language Model (Moonlight): Based on a lightweight MoE architecture with 6B total parameters and 2.8B activated parameters. Initialized from an intermediate checkpoint in the pretraining stage; the checkpoint had already processed 5.2T tokens of pure text data and activated a context length of 8K. Using a mixed training scheme, continued pretraining on 2.3T tokens of multimodal and pure text data.
• Pretraining Stage：
  • Independent ViT Training: Independently train the vision encoder to build robust visual feature extraction capabilities.
  • Joint training: It includes three stages: pretraining, cooldown, and long-context activation, simultaneously enhancing the model's language and multimodal capabilities.
  • Post-training stage: Conducted in 32K and 128K contexts respectively to further optimize model performance. Fine-tune activation and enhance long-thinking capabilities with Long-CoT, and further improve the model's reasoning ability based on reinforcement learning.
• Performance optimization: Based on a 128K context window, it performs precise information retrieval in long texts and long videos. The native-resolution encoder MoonViT maintains high accuracy in ultra-high-resolution vision tasks with lower computational overhead for users. Based on Long CoT activation and reinforcement learning, Kimi-VL-Thinking performs deeper reasoning on complex tasks.

Kimi-VL's performance advantages

• High intelligence: Kimi-VL performs excellently in multimodal reasoning and multi-step Agent tasks, and its text processing capability is in no way inferior to pure text language models. On benchmarks such as MMMU, MathVista, and OSWorld, Kimi-VL achieves impressive results without relying on "long thinking" capability, demonstrating outstanding intelligence.
• Long context: Kimi-VL has an ultra-long 128K context window. When handling long video and long document tasks, such as the LongVideoBench and MMLongBench-Doc benchmarks, it performs far better than other models of the same level, supports precise retrieval and understanding of massive information, and provides more comprehensive context support for complex tasks.
• Stronger visual capabilities: Compared with other open-source vision-language models, Kimi-VL demonstrates comprehensive and significant competitive advantages in multiple visual scenarios, including visual perception, visual world knowledge, OCR, and high-resolution screenshot parsing. Whether capturing details in complex images or deeply understanding visual information, Kimi-VL can complete tasks accurately and efficiently.

Kimi-VL's project addresses

• GitHub repository：https://github.com/MoonshotAI/Kimi-VL
• HuggingFace model library：https://huggingface.co/collections/moonshotai/kimi-vl
• Technical paper：https://github.com/MoonshotAI/Kimi-VL/blob/main/Kimi-VL.pdf

Kimi-VL's application scenarios

• Intelligent customer service: Used in multi-turn conversations, answering user questions, and supporting interactions that combine images and text.
• Educational tutoring: Assists students in learning, providing illustrated answers and teaching materials.
• Content creation: Generates image-text and video content, assisting video editing and creative generation.
• Medical assistance: Analyzes medical images, providing preliminary diagnostic suggestions and health consultations.
• Enterprise office work: Processes long documents, extracts key information, and supports intelligent meeting services.