SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

Phi-4-reasoning-vision-15B - Microsoft's Open-Source Multimodal Reasoning Model

Author: AI小集 · Original source: AI工具集

What is Phi-4-reasoning-vision-15B

Phi-4-reasoning-vision-15B is Microsoft's open-source 15-billion-parameter multimodal reasoning model, built onPhi-4-Reasoninga language model andSigLIP-2a vision encoder. The model adopts a "hybrid reasoning" design that can automatically determine when deep thinking is needed (math/science problems) or when a quick response suffices (OCR/description tasks). With only 200 billion tokens of training data, it achieves a balance between accuracy and efficiency. The model is particularly good at GUI understanding, mathematical reasoning, and document analysis, with inference speeds more than 10 times faster than comparable large models.

Phi-4-reasoning-vision-15B

Key Features of Phi-4-reasoning-vision-15B

  • General Visual Understanding: The model can analyze image content to generate descriptions, answer questions about images, and recognize objects, scenes, and text information.
  • Document and Chart Parsing: The model can read and understand various documents, receipts, invoices, and complex charts, extracting key information for quantitative analysis.
  • Mathematical and Scientific Reasoning: The model excels at solving math and science problems containing formulas and charts, and can recognize handwritten content and present the complete problem-solving process.
  • GUI Interaction and Screen Understanding: The model can identify interactive elements in computer and phone interfaces, provide precise bounding box coordinates, and support automated operation agents.
  • Multi-Image Sequence Analysis: The model can process multiple images, analyze temporal sequence changes, and understand the correlations and evolution trends between images.
  • Adaptive Reasoning Mode: Automatically selects direct answering or deep thinking based on task complexity; users can force a specific reasoning method through special tokens.

Technical Principles of Phi-4-reasoning-vision-15B

  • Architecture Design: It adopts a Mid-fusion architecture, using the SigLIP-2 Naflex dynamic-resolution vision encoder to process image inputs, generating visual tokens that are projected into the embedding space of the Phi-4-Reasoning language model, enabling cross-modal joint reasoning.
  • Hybrid Reasoning Mechanism: The model inherits from a language model backbone with reasoning capabilities and learns through supervised fine-tuning to distinguish between tasks requiring reasoning (math/science) and perceptual tasks (OCR/description). Training data is mixed with 20% reasoning samples and 80% non-reasoning samples, allowing the model to adaptively choose the depth of reasoning.
  • High-Resolution Processing: The model uses a SigLIP-2 Naflex variant that supports dynamic resolution, up to 3600 visual tokens, equivalent to native 720p resolution, significantly improving perception accuracy for high-density information interfaces and small interactive elements.
  • Data Strategy: The model primarily uses rigorously filtered open-source data, with quality improved through manual review, error correction, and synthetic generation, supplemented by domain-specific math and GUI datasets. It achieves performance competitive with trillion-token models using only 200 billion tokens.

Project Addresses for Phi-4-reasoning-vision-15B

  • Project Website：https://www.microsoft.com/en-us/research/blog/phi-4-reasoning-vision-and-the-lessons-of-training-a-multimodal-reasoning-model/
  • GitHub Repository：https://github.com/microsoft/phi-4-reasoning-vision-15B
  • HuggingFace Model Hub：https://huggingface.co/microsoft/Phi-4-reasoning-vision-15B
  • Technical Paper：https://www.microsoft.com/en-us/research/wp-content/uploads/2026/03/Phi-4-reasoning-vision-15B-Tech-Report.pdf

Application Scenarios of Phi-4-reasoning-vision-15B

  • Educational Assistance: Students photograph math homework or physics charts; the model recognizes handwritten content, checks for errors in solution steps, and provides the correct derivation process.
  • Smart Office: The model supports automatic parsing of invoices and receipts, extraction of information such as amounts, dates, and items, execution of split-payment calculations, and generation of structured data output.
  • Interface Automation: As a foundational model for computer-use agents, it recognizes buttons, input fields, and menu elements on the screen, enabling cross-platform automated operations and workflow execution.
  • Mobile Device Assistance: The model supports understanding mobile app interfaces, helping users locate feature entry points, fill out forms, or perform complex multi-step tasks.
  • Document Analysis: The model can process scanned PDFs, charts, and reports, performing information extraction, trend analysis, and cross-document content comparison.