What is Kwai Keye-VL
Kwai Keye-VL is a multimodal large language model independently developed by Kuaishou. Based on the Qwen3-8B language model, it integrates a SigLIP-initialized vision encoder and supports dynamic resolution input. The model can deeply fuse and process multimodal information such as text, images, and videos. With an innovative adaptive interaction mechanism and dynamic reasoning capabilities, it is dedicated to creating a more intelligent and comprehensive new paradigm of multimodal interaction for users. The model performs excellently in video understanding, complex visual perception, logical reasoning, and other areas, especially scoring an impressive 140 points on the 2025 national college entrance examination mathematics paper. The model has been officially open-sourced, providing strong support for multimodal research and applications.
Main features of Kwai Keye-VL
- Video understanding: Deeply understand short video content, such as analyzing information including scenes, people, and actions in videos, to generate descriptions, tags, or recommend related content for videos.
- Image recognition and description: Automatically parse image details, recognize objects, scenes, etc. in images, and generate accurate descriptions.
- Logical reasoning: Performs excellently in complex logical reasoning tasks, such as solving mathematical problems, conducting scientific reasoning, and more.
- Multimodal interaction: Supports processing information in multiple modalities such as text, images, and videos, and performs effective interaction and fusion between modalities.
- Intelligent creation: Based on understanding multimodal information, assists users in content creation, such as generating copy, scripts, creative plans, and more.
Technical principles of Kwai Keye-VL
- Model architecture: Based on the Qwen3-8B language model, integrating a SigLIP-initialized vision encoder. Supports dynamic resolution input, splitting images into 14×14 patches according to the original aspect ratio, and integrating visual features through an MLP layer. Uses 3D RoPE (rotary position embedding) to uniformly process text, images, and videos, aligning based on position encoding and timestamps to precisely capture temporal changes in videos.
- Pre-training strategy: Continuously pre-train the vision encoder to adapt to internal data distribution and support dynamic resolution input. Freeze the backbone model, train only a lightweight MLP adapter, and efficiently establish robust image-text/video-text alignment relationships at extremely low cost. Unlock all model parameters, conduct multi-task joint training, and comprehensively improve the model's overall visual understanding ability. Fine-tune with selected high-quality data to further improve the model's fine-grained understanding and discrimination ability. Explore homogeneous-heterogeneous fusion technology, averaging parameters to fuse annealing-trained models with different data ratios, preserving multi-dimensional capabilities while reducing model bias and enhancing model robustness.
- Post-training strategy：
- No-Reasoning Training): Uses 5 million high-quality multimodal VQA data entries. Data diversity is ensured by the task system built by the self-developed TaskGalaxy solution (including 70,000 types of tasks), and data quality is ensured by AI screening of difficult samples and manual annotation. Combines open-source data with self-built preference data; the latter collects SFT error samples as question materials, generates answer pairs with Qwen2.5VL 72B and the SFT model, and obtains them through manual ranking.
- Reasoning Training: Training data that mixes four reasoning modes, achieving zero-based activation of the model's chain-of-thought capabilities and enabling it to initially master the reasoning paradigm of human step-by-step thinking. On the basis of cold start, the GRPO algorithm is used for mixed-mode reinforcement learning, and based on an innovative dual-track reward mechanism (synchronously evaluating result correctness and process consistency), it deeply optimizes comprehensive capabilities such as multimodal perception, mathematical reasoning, short video understanding, and agent collaboration, significantly improving the model's reasoning ability. Based on the MPO algorithm, multiple rounds of iteration are performed on good-versus-bad data pairs, fundamentally solving the problems of content repetition collapse and logical discontinuity, and ultimately giving the model the adaptive ability to intelligently select deep reasoning modes according to question complexity, achieving a dual breakthrough in performance and stability.
Project address of Kwai Keye-VL
- Project official website：https://kwai-keye.github.io/
- GitHub repository：https://github.com/Kwai-Keye/Keye/tree/main
- HuggingFace model library：https://huggingface.co/Kwai-Keye
Application scenarios of Kwai Keye-VL
- Video content creation: Help short video creators quickly generate titles, descriptions, and scripts, improving creation efficiency.
- Intelligent customer service: Based on multimodal interaction (text, voice, images), provide users with intelligent customer service and enhance user experience.
- Educational tutoring: Provide students with personalized learning tutoring, including homework solutions and explanations of knowledge points, to support learning.
- Advertising and marketing: Generate attractive copy and scripts for advertisers, improving advertising effectiveness.
- Medical assistance: Assist doctors in analyzing medical images, provide preliminary diagnostic suggestions, and improve medical efficiency.