What is Light-R1
Light-R1 is an AI model open-sourced by 360 Zhinao, focused on long chain-of-thought reasoning in the mathematics domain, specifically Light-R1-32B. The model is based on Qwen2.5-32B-Instruct and trained with 70,000 math data samples and a two-stage curriculum learning approach (SFT+DPO), achieving performance from scratch that surpasses DeepSeek-R1-Distill-Qwen-32B. On the AIME24 test, Light-R1 scored 76.6, significantly higher than DeepSeek-R1-Distill's 72.6. The model has low training cost, requiring only 12 H800 machines running for 6 hours, at a cost of about $1,000. The model supports full open-source release, including the model, dataset, training framework, and evaluation code, promoting the development of the open-source community and providing a reference for low-cost training of domain-specialized models.
Main Features of Light-R1
- Efficient Mathematical Problem Solving: Can quickly and accurately solve complex mathematical problems, including but not limited to algebra, geometry, probability, and other fields.
- Improved Reasoning Ability: Possesses strong logical reasoning ability and supports handling long chain-of-thought problems.
- Generalization Ability: Demonstrates generalization ability in other fields (such as logical reasoning and language understanding).
- Low-Cost Training and Deployment: Achieves high performance at extremely low cost, suitable for users or enterprises with limited resources for rapid deployment and application.
Technical Principles of Light-R1
- Base Model and Starting Point: The model is developed based on Qwen2.5-32B-Instruct, achieving performance improvement from scratch to surpassing DeepSeek-R1-Distill.
- Curriculum Learning：
- SFT（Supervised Fine-Tuning): Screens difficulty-graded data and conducts supervised fine-tuning in two stages. The first stage uses 70,000 data samples, and the second stage screens the 3,000 most difficult data samples for further fine-tuning.
- DPO（Direct Preference Optimization）: On the basis of SFT, optimizes the model's output quality based on multiple sampling and the construction of preference pairs.
- Data Processing and Deduplication: The training data comes from multiple open-source math datasets (such as OpenR1-Math-220k, OpenThoughts-114k, etc.), and undergoes strict data deduplication processing to avoid the impact of test data leakage on model performance.
- Model Fusion: The final Light-R1-32B is obtained by fusing the SFT stage 2, DPO, and another DPO version of the model. This further improves the model's performance and stability.
- Training Framework and Optimization: Uses the 360-LLaMA-Factory training framework, supporting sequence parallelism and efficient distributed training. Based on the optimized training process, Light-R1 can complete training in only 6 hours on 12 H800 machines.
Project Address of Light-R1
- GitHub Repository：https://github.com/Qihoo360/Light-R1
- HuggingFace Model Library：https://huggingface.co/collections/qihoo360/light-r1
Application Scenarios of Light-R1
- Education Field: As a mathematics learning tool, it helps students solve complex problems and provides problem-solving steps and ideas, suitable for mathematics competitions and daily learning.
- Scientific Research and Academia: Assists mathematical research and interdisciplinary problem solving, such as physical modeling and engineering optimization.
- Enterprise Applications: Used for solving complex problems such as data analysis, risk assessment, and supply chain optimization.
- Software Integration: Integrated into intelligent assistants and mathematics software to enhance reasoning and problem-solving functions.
- Open Source and Developers: Supports developers in customizing and extending the model, promoting the development of the open-source community.