s1

Purpose

s1 is an open-source project for studying "test-time scaling." Its core goal is to use very few examples (1,000) combined with the budget forcing technique to achieve strong reasoning performance on reasoning tasks comparable to o1-preview, and to publicly release a reproducible training and evaluation recipe.

Main capabilities

• Provides a complete resource entry point for the paper "s1: Simple test-time scaling."
• Released models: s1.1-32B (current) and s1-32B (older version).
• Released datasets: s1K-1.1 (current) and s1K (older version), plus s1-prob, s1-teasers, and Full 59K for data ablation.
• Inference supports vLLM (including budget forcing) and transformers.
• Provides evaluation scripts in eval/, synthetic data creation scripts in data/, and training scripts in train/.
• Provides the training script train/sft.py and the accompanying train/sft*sh and train/launch.sh (SLURM cluster).
• Provides the evaluation command eval/commands.sh and the statistics script eval/compute_sample_stats.py.
• Supports REBASE experiments, with commands located in eval/rebase/run.sh.
• Provides the complete pipeline for reproducing s1K (data collection, Gemini trajectory generation, Qwen answer generation, featurization, filtering, grading).
• Figures can be reproduced via visuals/visuals.ipynb and visuals/s1.fig.

Usage

• Inference: install and run vLLM (supports budget forcing), or install transformers and torch and then run the corresponding scripts.
• Training: use train/sft.py, invoked via train/sft*sh; on a SLURM cluster, it can be launched with train/launch.sh (the file must be modified according to the cluster configuration). Training s1-32B/s1.1-32B is recommended with 16 H100 GPUs, i.e., 2 nodes with 8 each. For s1.1, set the block size to 20000 to avoid OOM; if OOM occurs with 8 GPUs, you can add --gradient_checkpointing=True to the script.
• Evaluation: clone and modify lm-evaluation-harness (commit 4cec66e4e468d15789473d6d63c3a61a751fa524); see eval/commands.sh for commands; for AIME24, use the aime24_nofigures results.
• Reproduce s1K: run data/collect_data.py, then run data/fix_gpqa.py and data/add_aime.py; use data/gemini.py to generate Gemini trajectories; use data/bulk_inference.py to generate Qwen answers; run data/featurization.py to add features; perform final filtering via data/filter.ipynb; if grading is needed, use data/grading.ipynb. In the relevant files, replace the simplescaling and qfq organization names with organizations you own.

Applicable scenarios

• Researchers studying test-time scaling and budget forcing methods.
• Developers who need to reproduce the training and evaluation pipeline of the s1 series models.
• Teams that want to explore training recipes for strong reasoning models under limited data.

Limitations

• The official materials do not provide specific information on fees, pricing, or commercial use terms.
• The official materials do not list a complete inventory of all supported platforms or deployment environments.
• Training and evaluation involve many script modifications and organization name replacements; the official materials do not provide a one-click deployment solution.
• When using vLLM with budget forcing, a "Token id XXXXX is out of vocabulary" error may be encountered at temperature 1. The official workaround is to comment out the ValueError check on the line if max_input_id > tokenizer.max_token_id: in vllm/engine/llm_engine.py.

Official resources

• Paper:https://arxiv.org/abs/2501.19393
• Model:https://hf.co/simplescaling/s1.1-32B（旧版：https://hf.co/simplescaling/s1-32B）
• Data:https://hf.co/datasets/simplescaling/s1K-1.1（旧版：https://hf.co/datasets/simplescaling/s1K）
• Other data:https://hf.co/datasets/simplescaling/s1-prob ，https://hf.co/datasets/simplescaling/s1-teasers ，https://hf.co/datasets/simplescaling/data_ablation_full59K
• Evaluation results:https://hf.co/datasets/simplescaling/results
• Model used for reproducing REBASE:https://huggingface.co/simplescaling/step-conditional-control-old
• Old dataset:https://huggingface.co/datasets/simplescaling/s1K-step-conditional-control-old
• Old evaluation data:https://huggingface.co/datasets/Maxwell-Jia/AIME_2024