What is Skywork-OR1
Skywork-OR1 (Open Reasoner 1) is a series of open-source high-performance reasoning models launched by Kunlun Wanwei. The models break through the capability bottlenecks of large models in logical reasoning and complex task solving. Skywork-OR1 includes three models: Skywork-OR1-Math-7B is a mathematical reasoning model with powerful mathematical problem-solving capabilities; Skywork-OR1-7B-Preview is a general-purpose model with both mathematical and coding capabilities; Skywork-OR1-32B-Preview is the flagship version, suitable for tasks of higher complexity, with stronger reasoning capabilities. In terms of performance, the Skywork-OR1 series performs excellently on multiple benchmarks. For example, on the AIME24 and AIME25 mathematical datasets, Skywork-OR1-Math-7B achieved high scores of 69.8% and 52.3% respectively, far exceeding mainstream models of the same scale. In competitive programming tasks, Skywork-OR1-32B-Preview's performance on the LiveCodeBench dataset is close toDeepSeek-R1(671B parameters), demonstrating excellent cost-effectiveness.
Main Features of Skywork-OR1
- Logical Reasoning Ability: Possesses powerful logical reasoning ability, handling complex logical relationships and multi-step reasoning tasks.
- Programming Task Support: Supports generating high-quality code, supporting multiple programming languages.
- Code Optimization and Debugging: Optimizes and debugs code, improving code readability and execution efficiency.
- Adaptation to Multi-Domain Tasks: Possesses general reasoning ability, supporting handling complex tasks in other domains.
- Multi-Turn Dialogue and Interaction: Supports multi-turn dialogue, gradually solving problems based on contextual information, providing a more coherent reasoning process.
Technical Principles of Skywork-OR1
- High-Quality Datasets: Based on high-quality mathematical datasets such as NuminaMath-1.5 (about 896,000 problems), high-difficulty subsets such as AIME and Olympiads are selected, totaling about 110,000 math problems. Mainly based on LeetCode and TACO data, after strict screening and deduplication, retaining problems with complete unit tests and verified correctness, obtaining 13.7K high-quality code problems.
- Data Preprocessing and Filtering: Each problem undergoes multiple rounds of sampling to verify answers, eliminating problems that are "all correct" or "all wrong" to avoid the impact of invalid data on training. Combining human review and LLM automatic problem-judging mechanisms, problems with unclear semantics, incomplete information, formatting errors, or irrelevant content are cleaned up.
- Training Strategy: Model training is based on GRPO, with multi-stage training gradually increasing the context window length to enhance the model's long-chain thinking ability. Offline and online filtering are performed before and during training respectively, dynamically eliminating invalid samples to ensure the validity and challenge of training data. During reinforcement learning sampling, a high sampling temperature (τ=1.0) is used, based on an adaptive entropy control mechanism, enhancing the model's exploration ability and avoiding falling into local optima too early.
- Loss Function Optimization: The KL loss term is removed during training, allowing the model to fully explore and optimize its reasoning ability. The policy loss is averaged over all tokens in the training batch, improving the consistency and stability of the optimization process.
- Multi-Stage Training: Based on multi-stage training, the context window length is gradually expanded, allowing the model to efficiently complete tasks within limited tokens and gradually master complex long-chain thinking abilities. In the early stage of multi-stage training, truncated samples are handled based on specific strategies to ensure the model rapidly improves performance when entering the next stage.
Performance of Skywork-OR1
- Mathematical Reasoning Tasks：
- The general models Skywork-OR1-7B-Preview and Skywork-OR1-32B-Preview achieve state-of-the-art performance at the same parameter scale on the AIME24 and AIME25 datasets, demonstrating strong mathematical reasoning capabilities.
- The specialized model Skywork-OR1-Math-7B achieves high scores of 69.8 and 52.3 on AIME24 and AIME25 respectively, far surpassing current mainstream 7B-level models, highlighting its professional advantages in advanced mathematical reasoning.
- Skywork-OR1-32B-Preview surpasses QwQ-32B on all benchmarks and is essentially on par with R1 on AIME25.
- Competitive Programming Tasks：
- The general models Skywork-OR1-7B-Preview and Skywork-OR1-32B-Preview achieve state-of-the-art performance at the same parameter scale on the LiveCodeBench dataset.
- Skywork-OR1-32B-Preview's code generation and problem-solving capabilities approach those of DeepSeek-R1 (with a parameter scale of 671B), achieving excellent cost-effectiveness while compressing model size, reflecting the advanced nature of its training strategy.
- Skywork-OR1-Math-7B Performance：
- The training accuracy curve on AIME24 shows steady performance improvement. The model reaches 69.8% and 52.3% on AIME24 and AIME25 respectively, surpassing OpenAI-o3-mini (low) and achieving state-of-the-art performance at the current size.
- On Livecodebench, it improves from 37.6% to 43.6%, a significant improvement over the baseline model, demonstrating the good domain generalization of the training method.
Skywork-OR1 Project Address
- Project Official Website：https://capricious-hydrogen-41c.notion.site/Skywork-Open-Reasoner
- GitHub Repository：https://github.com/SkyworkAI/Skywork-OR1
- HuggingFace Model Hub：https://huggingface.co/collections/Skywork/skywork-or1
Skywork-OR1 Application Scenarios
- Mathematics Education: Assist students in solving problems, provide ideas and steps, and help teachers prepare lessons.
- Scientific Research Assistance: Help researchers explore complex models, verify conjectures, and derive formulas.
- Programming Development: Generate code frameworks, optimize code, assist debugging, and improve development efficiency.
- Data Analysis: Support decision-making in fields such as finance and business, predict trends, and assess risks.
- AI Research: Serve as a research platform to advance improvements in reasoning model architectures and algorithms.