What is OpenAI o3
The OpenAI o3 model is a new-generation reasoning model following o1. It is OpenAI's first model to integrate image reasoning into the chain of thought, enabling it to think directly about images within the chain of thought. Users can upload blurry, upside-down, or low-quality images, and the model can still interpret them and perform operations such as cropping, zooming in, and rotating. o3 can intelligently select and combine various built-in tools of ChatGPT, such as web search, Python code execution, and file analysis, to solve complex problems. o3 performs excellently in fields such as programming, mathematics, and science, setting new records on multiple benchmark tests. For example, in the Codeforces programming competition test, o3 scored over 2700 points, ranking among the global top 200. OpenAI conducted comprehensive safety evaluations and training for o3, adding refusal prompts for biothreats, malware generation, and jailbreak attacks.
Main features of OpenAI o3
- Image thinking: o3 is the first to directly integrate images into the chain of thought, enabling reasoning with pictures. Users can upload blurry, upside-down, or low-quality images, and the model can still interpret them and perform operations such as cropping, zooming in, and rotating.
- Multimodal processing: o3 supports functions such as web search, file analysis, Python code execution, deep reasoning over visual input, and image generation. This enables excellent performance when handling multimodal tasks, allowing it to understand and solve problems more comprehensively.
- Tool calling: o3 can autonomously select and use tools to quickly generate detailed and thoughtful answers. For example, it can call search engines to obtain information, use Python for data analysis, and even generate images.
- Efficient reasoning: o3 has stronger performance at the same latency and cost, making it suitable for complex queries. It has set new records in multiple benchmark tests, including fields such as programming, mathematics, science, and visual perception.
- Reinforcement learning: o3 improves its reasoning ability through large-scale reinforcement learning. This training method enables the model to think about and solve problems more effectively when facing complex issues.
- Open-source tool: OpenAI has also launched Codex CLI, a lightweight programming agent that runs in the terminal. It supports multimodal input and local code execution, and users can perform efficient programming via the command line.
- Safety: o3 has made significant improvements in safety, rebuilding the safety training dataset and enhancing defenses against biothreats, malware generation, and jailbreak attacks.
- Cost efficiency: o3 costs $10 per million input tokens and $40 per million output tokens. Compared with previous-generation models, o3 has stronger performance at the same latency and cost.
- Outstanding programming performance: It achieved an ELO score of 2727 on the CodeForces programming competition platform, surpassing top programmers, and supports code generation and execution for complex tasks, automatically optimizing code logic and improving development efficiency.
- Transparent reasoning path: It provides a clear reasoning process, can display the logical thinking and intermediate conclusions of each step, and enhances the credibility and interpretability of decisions.
Technical principles of OpenAI o3
- Hybrid reasoning framework: o3 adopts a hybrid reasoning framework that combines neuro-symbolic learning with probabilistic logic. It integrates the powerful pattern recognition capabilities of neural networks with the logical rigor of symbolic reasoning, enabling the model to better handle structured data and knowledge. Probabilistic logic allows the model to make reasonable inferences when faced with vague or incomplete information.
- Multi-path Reasoning and Search Mechanism: o3 implements multi-path reasoning by generating multiple possible paths during the reasoning process, evaluating and selecting among these paths to find the optimal solution. Similar to Monte Carlo Tree Search (MCTS), it can effectively utilize computational resources and improve the model's performance on complex tasks.
- Problem Decomposition and Iterative Optimization: o3 can decompose complex problems into smaller, more manageable parts, and analyze and solve them step by step. This "divide and conquer" strategy enables the model to perform excellently on multi-step reasoning tasks. o3 has the capability for iterative optimization, continuously improving and refining answers through multiple rounds of reasoning.
- Powerful Context Memory: o3 has an extended memory function that can retain contextual information during long interactions. When handling tasks with strong continuity, the model can better understand and respond to previous conversations or data, providing users with more coherent and accurate answers.
- Optimized Model Architecture: o3 adopts a more advanced multi-layer transformer architecture, significantly enhancing the model's ability to process long text sequences and capture complex semantic relationships and logical structures in text.
- Reinforcement Learning and Training: o3 extensively uses reinforcement learning during training, continuously optimizing the model's reasoning ability through interaction with the environment. This training approach enables the model to think and solve problems more effectively when facing complex issues.
- Efficient Inference and KV Cache: o3 adopts efficient KV cache technology during the inference stage, reducing repeated computation and improving inference efficiency by storing and reusing intermediate results.
- Chain-of-Thought Search and Execution: The o3 model overcomes the limitations of traditional large language models when handling new problems by generating and executing its own programs. The program itself (the chain of thought) becomes a concrete embodiment of knowledge recombination.
- Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL): OpenAI used supervised fine-tuning and reinforcement learning from human feedback for safety training. It supports the model in learning ideal behaviors from examples and trains the model through reinforcement learning to use its chain of thought more effectively.
- Deliberative Alignment: It is a new safety evaluation method and a new paradigm that directly teaches the model safety specifications. By reasoning about the user's input intent, it greatly improves the model's ability to identify potentially unsafe requests.
- Adaptive Thinking Time: The o3 model supports three reasoning time modes: low, medium, and high. Users can flexibly adjust the model's thinking time according to task complexity to achieve optimal performance.
Benchmarks for OpenAI o3
- InARC-AGItest: o3 achieved under the high reasoning capability setting87.5%score, and under the low reasoning capability setting, its score is also as high as o1's3 times。
- In programming competitionCodeforces: o1's score is 1891, while o3 can reach under the high reasoning setting2727score, and the score under the low reasoning setting also exceeds o1.
- AIME 2024：In the math benchmark testAIME 2024, o3's accuracy reaches96.7%
- EpochAI Frontier Math：o3 has set a new record in theEpochAI Frontier Mathjointly launched by Terence Tao and more than 60 mathematicians worldwide, billed as the industry's strongest math benchmark, with a score of25.2. No other model exceeded 2.0.
| Test Type | o3 Performance | Human Expert Level | Notes |
|---|---|---|---|
| ARC-AGI Test | 87.5% | 85% | The score under the low-reasoning setting is also as high as o1's3 times |
| CodeForces Elo Rating | 2727 | – | Surpasses 99.99% of human programmers; o1's score is 1891 |
| AIME 2024 Math Competition | 96.7% | – | Almost a perfect score |
| GPQA Diamond Test | 87.7% | 70% | Significantly exceeds the average level of human experts |
| EpochAI Frontier Math | 25.2% | – | Other models did not exceed 2.0% |
Differences Between OpenAI o3 and o1
- Performance Improvement: According to the SWE-bench Verified code generation evaluation benchmark, o3's accuracy score is 71.7, exceeding o1's 48.9 and o1 preview's 41.3. In the 2024 AIME math competition problem test, o3's accuracy score is 96.7, exceeding o1 and o1 preview's 83.3 and 56.7.
- Cost: In low-compute mode, o3 costs $17–20 to complete each ARC-AGI task; in high-compute mode, completing each task costs thousands of dollars.
- Safety and Alignment: OpenAI is using a new technique called "deliberative alignment" to make models such as o3 comply with its safety principles. o3 is trained to "think" before responding, can reason about tasks and plan ahead, and execute a series of actions over a longer period to help find solutions.
OpenAI o3 Project Address
- Project Official Website：https://openai.com/index/introducing-o3-and-o4-mini/
How to Use OpenAI o3
- For Ordinary Users: Via ChatGPT access: ChatGPT Plus, Pro, and Team users can see o3, o4-mini, and o4-mini-high in the model selector, replacing the previous o1, o3-mini, and o3-mini-high. ChatGPT Enterprise and Edu users will gain access one week later. Free users can try o4-mini by selecting "Think" before submitting a query. Rate limits across all plans remain unchanged from the previous set of models.
- For Developers: Use via API: Developers can use o3 and o4-mini through the Chat Completions API and the Responses API.
Application Scenarios of OpenAI o3
- Mathematical Reasoning and Education: The o3 model performs excellently on complex math problems. o3 is very suitable for the education field, helping students solve math problems and providing solution ideas and methods.
- Programming and Software Development: Supports code generation and execution for complex tasks, automatically optimizes code logic, and improves development efficiency. o3 can serve as a programming assistant to help developers write, debug, and optimize code.
- Scientific Research and Data Analysis: o3 is suitable for data analysis and problem modeling in scientific research work, helping scientists handle complex scientific problems.
- Multimodal Problem Solving: The o3 model can handle mixed input of text and images, providing strong support for multimodal reasoning scenarios, such as visual reasoning and cross-modal problem solving. o3 can play a role in scenarios that require combining visual information and text information, such as image recognition and description generation.
- Transparent Reasoning Path: o3 provides a clear reasoning process, can show the logic and intermediate conclusions of each step, and enhances the credibility and interpretability of decisions. It is very useful in application scenarios that require explaining and verifying the reasoning process, such as legal analysis, financial risk assessment, etc.
- Efficient multitasking: o3 supports long-context input, can handle complex multi-step instructions, and is suitable for programming, scientific, and multimodal problem-solving scenarios.