What is Seed-Thinking-v1.5
Seed-Thinking-v1.5 is a reasoning intelligence model launched by ByteDance, adopting a Mixture-of-Experts (MoE) architecture, with a total parameter count of 200B and 20B parameters activated each time. The model performs excellently in multiple benchmarks, such as scoring 86.7 in the AIME 2024 test, achieving 55.0 in the pass@8 metric in the Codeforces evaluation, and scoring 77.3 in the GPQA test. In non-reasoning tasks, its win rate is 8% higher than DeepSeek R1, demonstrating broad applicability. The development of Seed-Thinking-v1.5 involves carefully curated training data, an advanced reinforcement learning framework, a two-layer reward system, and efficient infrastructure. The model will be available for users to experience through the Volcano Engine open interface on April 17.
Main features of Seed-Thinking-v1.5
- Outstanding reasoning ability: Seed-Thinking-v1.5 performs excellently in multiple authoritative benchmarks, for example, scoring 86.7 in the AIME 2024 test, reaching 55.0% in the pass@8 metric in the Codeforces evaluation, and scoring 77.3 in the GPQA test. This indicates that it has strong reasoning ability in the STEM (science, technology, engineering, and mathematics) fields as well as in programming.
- Broad generalization ability: The model also performs excellently in non-reasoning tasks, with a win rate 8% higher than DeepSeek R1, showing its advantages in handling complex user scenarios.
- Efficient infrastructure: To support large-scale training, Seed-Thinking-v1.5 adopts the HybridFlow programming model and the streaming reasoning system (SRS), optimizing training efficiency through a three-layer parallel architecture (tensor/expert/sequence parallelism).
Technical principles of Seed-Thinking-v1.5
- Mixture-of-Experts (MoE) architecture: Seed-Thinking-v1.5 adopts a Mixture-of-Experts (MoE) architecture, with a total parameter count of 200B and 20B parameters activated each time. While maintaining high performance, it significantly reduces the consumption of computing resources and improves the model's efficiency.
- Reinforcement learning algorithms
- VAPO and DAPO frameworks: To solve the instability problem in reinforcement learning training, the research team proposed two major frameworks, VAPO (for Actor-Critic) and DAPO (for Policy Gradient). These two methods can provide robust training trajectories and effectively optimize reasoning models.
- Reward modeling: The team designed two reward modeling schemes, Seed-Verifier and Seed-Thinking-Verifier. Through detailed reasoning processes, Seed-Thinking-Verifier solves problems such as reward hacking, prediction uncertainty, and failure to handle edge cases.
- Data processing and enhancement
- Verifiable questions: Including mathematics, programming, and logic puzzles, ensuring that the model learns genuine reasoning ability through strict screening and enhancement (such as converting multiple-choice questions into fill-in-the-blank questions).
- Non-verifiable questions: Such as creative writing and dialogue, avoiding the model falling into local optimization by dynamically filtering low-variance samples.
- Data enhancement strategies: For example, using the model to generate candidate answers and combining manual verification to correct erroneous reference answers, improving data reliability.
- Distributed training infrastructure
- Hybrid parallel architecture: Combining tensor parallelism (TP), expert parallelism (EP), and context parallelism (CP) to support efficient large-scale training.
- Streaming generation system (SRS): Through asynchronous processing and dynamic resource scheduling, it improves the efficiency of long text generation by 3 times, solving the "tail-latency problem" of traditional synchronous frameworks.
- Automatic tuning system: Dynamically selects the optimal computing configuration based on real-time load, balancing memory and computing resources.
Project address of Seed-Thinking-v1.5
- Github repository：https://github.com/ByteDance-Seed/Seed-Thinking-v1.5
Performance of Seed-Thinking-v1.5
- Mathematical reasoning: Achieved a score of 86.7 on the AIME 2024 test, on par with OpenAI's o3-mini-high.
- Programming tasks: In the Codeforces evaluation, the pass@8 metric reached 55.0%, outperforming DeepSeek-R1.
- Scientific question answering: Achieved an accuracy of 77.3% on the GPQA test, approaching the level of top models.
- Non-reasoning tasks: In non-reasoning tasks, the win rate is 8% higher than DeepSeek R1, demonstrating strong generalization ability.
Application scenarios of Seed-Thinking-v1.5
- Scientific question answering: Seed-Thinking-v1.5 also has notable performance in scientific question answering. The model can understand and answer complex questions involving scientific concepts and principles, making it suitable for the fields of education and research.
- Creative writing: The model can generate cross-temporal dialogues, such as simulating the inner monologue of historical figures or narratives that integrate terminology from different fields. It has potential application value in fields such as content creation, advertising, and scriptwriting.
- Logical reasoning: Seed-Thinking-v1.5 has significant advantages when handling problems that require logical analysis and reasoning, and is suitable for scenarios that require logical judgment and analysis, such as legal analysis and market strategy planning.
- Educational assistance: The reasoning ability of Seed-Thinking-v1.5 can help students solve mathematics and science problems, provide feedback on programming exercises, and assist language learning.