What is Open-o3 Video
Open-o3 Video is an open-source video reasoning model jointly developed by Peking University and ByteDance, achieving precise video reasoning by integrating explicit spatiotemporal evidence (key timestamps and bounding boxes). Through the carefully curated STGR dataset and a two-stage SFT-RL training strategy, it achieves state-of-the-art performance on the V-STAR benchmark. Its non-agentic framework design efficiently handles complex spatiotemporal relationships and performs excellently in video reasoning tasks. The training process includes two stages: cold-start initialization and reinforcement learning, through which the model can better adapt to different video reasoning scenarios.
Main Features of Open-o3 Video
- Spatiotemporal Reasoning: It can integrate explicit spatiotemporal evidence, including key timestamps and bounding boxes, to perform precise video reasoning and effectively handle temporal and spatial relationships in videos.
- Dataset Curation and Training Strategy: It carefully curates the STGR dataset and adopts a two-stage SFT-RL training strategy, first performing cold-start initialization and then optimizing model performance through reinforcement learning, enabling it to perform excellently on the V-STAR benchmark.
- Non-Agentic Framework Design: It adopts a non-agentic framework to efficiently handle complex spatiotemporal relationships and improve the accuracy and efficiency of video reasoning.
- Open Source and Scalability: Its open-source nature allows researchers and developers to conveniently use and improve the model, promoting the development of video reasoning technology, and it has good scalability.
Technical Principles of Open-o3 Video
- Spatiotemporal Evidence Integration: By explicitly introducing key timestamps and bounding boxes as spatiotemporal evidence, it closely combines the video reasoning process with specific visual observations, making the model's reasoning more interpretable and reliable.
- Two-Stage Training Strategy: It adopts a two-stage training method combining cold-start initialization and reinforcement learning. The cold-start stage provides the model with basic spatiotemporal reasoning capabilities through supervised learning; the reinforcement learning stage further optimizes the model's reasoning performance through multiple reward mechanisms, improving the accuracy, temporal alignment, and spatial precision of answers.
- Dataset Curation: It carefully curates two high-quality datasets, STGR-CoT-30k and STGR-RL-36k, providing rich spatiotemporal annotations and reasoning traces for model training and solving the problem that existing datasets lack unified spatiotemporal supervision.
- Non-Agentic Framework Design: The model is built on a non-agentic framework, which can efficiently handle complex spatiotemporal relationships, avoiding the problems of information loss and low reasoning efficiency that agentic models may bring, and improving the overall efficiency and accuracy of video reasoning.
Project Address of Open-o3 Video
- Project Website：https://marinero4972.github.io/projects/Open-o3-Video/
- Github Repository：https://github.com/marinero4972/Open-o3-Video
- HuggingFace Model Library：https://huggingface.co/marinero4972/Open-o3-Video/tree/main
- arXiv Technical Paper：https://arxiv.org/pdf/2510.20579
Application Scenarios of Open-o3 Video
- Video Content Understanding: It can accurately understand and analyze key events and objects in videos, provide detailed reasoning and explanations for video content through spatiotemporal evidence, and help users better understand the core information of videos.
- Video Question Answering System: It can serve as a core component of a video question answering system, quickly locate relevant spatiotemporal segments in videos according to user questions, generate accurate and explanatory answers, and improve user experience.
- Video Editing and Creation: Provides assistance for video editing and creation, helping creators quickly find key elements and highlights in videos, and more efficiently carry out editing, special effects addition, and other creative activities.
- Intelligent Monitoring and Analysis: In the field of intelligent monitoring, it can analyze surveillance video in real time, quickly identify abnormal events and key objects, and provide detailed spatiotemporal evidence, helping to upgrade security monitoring with intelligence.
- Education and Training: In education and training, it can be used to analyze instructional videos, helping teachers and students better understand the teaching content, while also providing students with more targeted learning suggestions and feedback.
- Entertainment and Interaction: In the entertainment field, such as short video platforms and live streaming, it can provide users with a richer interactive experience, for example by generating interesting Q&A or challenges through video reasoning, enhancing user participation.