What is s1
s1 is a low-cost, high-performance AI reasoning model developed by research teams from Stanford University and the University of Washington. The model extracts reasoning capabilities from Google'sGemini 2.0 Flash Thinking Experimentalmodel through "distillation" technology. The researchers trained it using only 1,000 carefully curated questions and their answers, with a training cost of less than $50 and a training process lasting less than 30 minutes. The S1 model performs excellently in math and programming ability tests, comparable to top reasoning models such as OpenAI's o1 andDeepSeek R1.
Main features of s1
- Efficient reasoning capability: The S1 model focuses on reasoning about complex problems and performs excellently in the fields of mathematics and programming. It can solve highly difficult competition-level math problems, such as AIME (American Invitational Mathematics Examination) questions. The S1 model shows a significant improvement on competition math problems, exceeding OpenAI's o1-preview model by up to 27%.
- Low-cost training: The S1 model is trained using only 1,000 carefully curated questions and their reasoning traces. The training cost is extremely low, requiring less than $50 in cloud computing costs, and the training time is less than 30 minutes.
- Test-time Scaling: The S1 model dynamically adjusts the amount of computation at test time through budget forcing technology. By forcibly terminating the model's thinking process or appending a "Wait" instruction to extend the thinking time, the model can re-examine answers, correct erroneous reasoning steps, and improve reasoning performance.
- Open source and scalability: The S1 model's code, data, and training method have been open-sourced on GitHub, making it convenient for other researchers and developers to use and improve.
Technical principles of s1
- Dataset construction (s1K)
- Data sources: The S1 model's dataset s1K contains 1,000 high-quality questions, selected from multiple fields (such as mathematics, physics, chemistry, etc.), covering a variety of reasoning tasks.
- Screening criteria: Questions are screened using three criteria: difficulty, diversity, and quality. Difficulty is measured by model performance and reasoning trace length; diversity is classified by the domain to which the question belongs; quality is ensured by the accuracy of the data format and content.
- Final selection: The finally selected questions cover 50 different fields, ensuring the diversity and representativeness of the data.
- Supervised fine-tuning (SFT)
- Model selection: Qwen2.5-32B-Instruct is used as the base model, and the model performs excellently on math tasks.
- Training process: Supervised fine-tuning is performed on the s1K dataset. The training time is 26 minutes, using 16 NVIDIA H100 GPUs. During training, the model learns the mapping from questions to reasoning traces and answers.
- Budget Forcing
- Controlling the amount of computation at test time: The amount of computation is controlled by forcibly terminating or extending the model's thinking process at test time. Specific methods include:
- Forced termination: If the number of thinking tokens generated by the model exceeds a preset maximum limit, the thinking process is forcibly terminated and the model is made to output the answer directly.
- Appending "Wait": If the model is to be made to think for a longer time, a "Wait" instruction is appended after the current reasoning trace to prompt the model to continue exploring.
- Controlling the amount of computation at test time: The amount of computation is controlled by forcibly terminating or extending the model's thinking process at test time. Specific methods include:
- Evaluation of test-time scaling methods
- Evaluation metrics: Evaluate different test-time scaling methods through three metrics: Controllability (Control), Scalability (Scaling), and Performance (Performance).
- Method comparison: The S1 model compared multiple test-time scaling methods, including token-based control, step-based control, and category-based control. Ultimately, the budget forcing method performed best in controllability, scalability, and performance.
s1's project address
- Github repository：https://github.com/simplescaling/s1
- HuggingFace model library：https://huggingface.co/simplescaling/s1-32B
- arXiv technical paper：https://arxiv.org/pdf/2501.19393
s1's application scenarios
- Scientific problems: The S1 model can be applied to solve highly difficult scientific problems, such as competition-level problems in physics, chemistry, and biology.
- Intelligent tutoring system: The S1 model can serve as the core of an intelligent tutoring system, helping students solve complex mathematics and science problems, and providing detailed reasoning steps and explanations.
- Automatic question-answering system: The S1 model can be used in automatic question-answering systems, especially in scenarios requiring complex reasoning and multi-step thinking, such as solving highly difficult questions posed by users.
- Text generation: The S1 model can be used to generate high-quality text content in text generation tasks that require logical reasoning and complex structure.
- Intelligent customer service: The S1 model can be applied to intelligent customer service systems, solving complex user problems and providing more accurate and efficient answers.
- Data analysis: The S1 model can be used in data analysis and prediction tasks, in scenarios requiring reasoning and logical analysis.