What is Step-2 mini
Step-2 mini is a lightweight and ultra-fast large model launched by StepFun, developed based on the new-generation self-developed Attention architecture MFA. With only 3% of the parameter count, it retains over 80% of Step-2's performance, significantly improving generation speed and cost-effectiveness. When the model input is 4000 tokens, the average first-token latency is only 0.17 seconds, demonstrating extremely fast response capability. Step-2 mini adopts the MFA architecture, which saves nearly 94% of KV cache overhead compared to traditional multi-head attention architectures, greatly reducing inference costs.
Main features of Step-2 mini
- General task processing: It can handle a variety of general language tasks, such as text generation, question answering, translation, etc.
- Code generation and optimization: It performs outstandingly in code generation, able to understand user needs and generate executable code.
- Logical reasoning and mathematical problem solving: It has strong logical reasoning ability and can solve complex mathematical problems.
Technical principles of Step-2 mini
- Multi-matrix factorization attention mechanism (MFA) architecture: The MFA architecture is a new type of attention mechanism jointly developed by StepFun and Tsinghua University and other institutions. Through matrix factorization, it significantly reduces the key-value cache (KV Cache) usage in traditional attention mechanisms and lowers memory consumption. The MFA architecture adopts an aggressive low-rank factorization strategy, successfully maintaining extremely high parameter efficiency while expanding the number and dimensions of the model's attention heads.
- Reinforcement learning technology: Step-2 mini achieves a model that excels in both humanities and sciences through large-scale reinforcement learning training using the On-Policy reinforcement learning algorithm.
- High cost-effectiveness and fast response: While maintaining low computational cost, Step-2 mini responds extremely quickly, making it suitable for scenarios with high requirements for efficiency and cost.
Project address of Step-2 mini
- Project official website: VisitStepFun Open Platformto call the API interface.
Model pricing of Step-2 mini
- Price: Input 1 yuan/million tokens; output 2 yuan/million tokens.
Application scenarios of Step-2 mini
- Mathematical problem solving: Step-2 mini can construct reasonable reasoning chains to plan and solve complex mathematical problems step by step.
- Logical reasoning: In logical reasoning tasks, Step-2 mini can independently try multiple problem-solving approaches. After obtaining a preliminary answer, it asks itself whether there are other possibilities, ensuring that all effective solutions are enumerated.
- Data analysis: Step-2 mini can help researchers perform logical reasoning and data analysis, integrate interdisciplinary knowledge, and advance scientific research projects.
- Literature understanding: The model can understand and summarize scientific research literature and provide suggestions on key information and research directions.
- Code development: Step-2 mini assists programmers in efficiently developing code, providing code examples and logic analysis.
- Business decision-making: It provides managers with logical analysis and suggestions for business decisions and optimizes office processes.