What is Mini-o3
Mini-o3 is an open-source model jointly launched by ByteDance and the University of Hong Kong, designed specifically to solve complex visual search problems. Through reinforcement learning and image-based tools, the model can perform deep multi-turn reasoning, with reasoning turns extensible to dozens of rounds. By constructing challenging datasets, adopting an iterative data collection process, and proposing strategies such as over-turn masking, Mini-o3 significantly improves the model's reasoning capabilities and interaction turns. The model achieves state-of-the-art performance on multiple visual search benchmarks, and all code, models, and datasets are open-sourced, facilitating reproduction and further research.
Main Features of Mini-o3
-
Multi-turn Interactive Reasoning: Capable of deep multi-turn reasoning, with interaction turns extensible to dozens of rounds, solving complex visual search problems through step-by-step exploration and trial-and-error.
-
Diverse Reasoning Modes: Supports multiple reasoning modes, including depth-first search, trial-and-error, and goal maintenance.
-
Challenging Visual Search: In high-resolution images, even when targets are small and there are numerous distracting objects, it can accurately locate and identify targets.
-
Outstanding Performance: Achieves state-of-the-art results on multiple visual search benchmarks (such as VisualProbe, V* Bench, HR-Bench, MME-Realworld, etc.), demonstrating powerful visual reasoning capabilities.
-
Open Source: All code, models, and datasets are open-sourced, making it easy for researchers to reproduce and further research, advancing the development of related technologies.
Technical Principles of Mini-o3
-
Cold-start Supervised Fine-tuning (SFT): Through a small number of hand-crafted examples, uses a vision-language model (VLM) with in-context learning capability to generate high-quality, diverse multi-turn interaction trajectories.
-
Reinforcement Learning (RL): Based on the over-turn masking strategy, avoids penalties for interaction turns exceeding the limit, and can naturally extend to dozens of interaction rounds at test time.
-
Lower Down Max Pixels: By reducing the maximum number of pixels per image, increases the number of turns allowed in a single interaction, improving the ability to solve long-horizon problems.
-
Challenging Dataset (Visual Probe Dataset): Constructs a dataset containing thousands of visual search problems, with questions designed to encourage exploratory reasoning, helping the model learn complex reasoning patterns during training.
Mini-o3 Project Address
- Project Official Website：https://mini-o3.github.io/
- GitHub Repository：https://github.com/Mini-o3/Mini-o3
- HuggingFace Model Library：https://huggingface.co/Mini-o3/models
- arXiv Technical Paper：https://arxiv.org/pdf/2509.07969
Application Scenarios of Mini-o3
- E-commerce Scenarios: Helps users quickly find target products among massive product images. On a clothing e-commerce platform, users can search for similar styles of clothing by uploading an image.
- Smart Home: In a smart home environment, captures images through cameras to help users quickly find lost items, such as keys, remote controls, etc.
- Surveillance Video Analysis: Quickly locates and identifies specific targets in surveillance videos, such as finding specific persons or items in crowded places.
- Abnormal Behavior Detection: Analyze abnormal behavior in surveillance videos through multi-round reasoning, such as intrusion detection, abnormal activity recognition, etc.
- Navigation in complex scenes: In complex road conditions, multi-round visual reasoning helps autonomous driving systems better understand and plan routes, for example in scenes with occlusions or complex traffic signs.