SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

Step 3 - StepFun's latest multimodal reasoning model

Author: AI小集 · Original source: AI工具集

What is Step 3

Step 3 is the latest generation of foundation large models released by StepFun, designed for the reasoning era, combining high performance with extreme cost efficiency. It adopts a MoE architecture with 321B total parameters and 38B activated parameters, and is the first full-size, native multimodal reasoning model, with strong visual perception and complex reasoning capabilities, enabling efficient applications across multiple domains. Through the AFD distributed inference system and MFA attention mechanism, it achieves a significant improvement in inference efficiency. On domestic chips, inference efficiency can reach 3 times that of comparable models, and on NVIDIA Hopper architecture chips, throughput is increased by over 70%, significantly reducing inference costs. Step 3 will be officially open-sourced on July 31, providing the world's developers and enterprises with the strongest multimodal reasoning model.

Step 3

Main features of Step 3

  • Visual perception: Step 3 can accurately recognize and analyze complex information in images and videos, for example, it can still accurately restore content in menu recognition with severe glare.
  • Complex reasoning: Supports cross-domain complex knowledge understanding and cross-analysis of mathematical and visual information, for example, combining WeChat group chat records and shopping receipts to automatically calculate AA-style expense splitting.
  • Multimodal task processing: As a native multimodal model, Step 3 can handle tasks in multiple modalities such as language and vision, meeting the needs of diverse application scenarios.
  • Efficient inference: Through system architecture innovation, Step 3 performs excellently in inference efficiency. On domestic chips, inference efficiency can reach up to that of DeepSeek-R1's 300%, and on NVIDIA Hopper architecture chips, throughput is increased by over 70%。
  • Hardware-friendly: Step 3 is compatible with multiple hardware platforms, including mainstream and domestic chips, and can significantly reduce inference costs and improve resource utilization.

Technical principles of Step 3

  • MoE architecture: Step 3 adopts the MoE (Mixture of Experts) architecture, which is an efficient method for model parallelization. By decomposing the model into multiple "expert" modules and dynamically selecting the appropriate experts for computation based on the input, the MoE architecture can significantly reduce wasted computational resources while maintaining high performance.
  • AFD distributed inference system: Allocates the attention (Attention) and feed-forward network (FFN) computation tasks in the model to the most suitable hardware, improving overall efficiency.
    • Attention computation: A task that heavily consumes memory bandwidth, allocated to GPU clusters with large memory bandwidth.
    • FFN computation: A task that heavily consumes compute power, allocated to GPU clusters with strong compute power.
  • MFA attention mechanism: Optimizes arithmetic intensity, adapts to the performance characteristics of mainstream and domestic chips, and enables efficient inference across hardware platforms.

Project address of Step 3

  • Github repository：https://github.com/stepfun-ai/Step3

Application scenarios of Step 3

  •  Intelligent terminal Agent: Step 3 can be applied to various IoT devices, such as smart homes, smart wearables, etc., providing intelligent voice assistant and visual recognition functions.
  • Finance and economics: Step 3 can be used in scenarios such as financial risk assessment, intelligent customer service, and market analysis. Through multimodal data processing, the model can more accurately analyze market trends and user needs.
  • Content creation: Step 3 can assist content creators in generating creative copy, images, and video content. For example, it can combine visual and textual information to generate high-quality advertising copy or video scripts.
  • Visual Recognition: Step 3 can handle complex visual tasks, such as reflective menu recognition, image classification, object detection, etc.
  • Complex Reasoning: Step 3 supports complex cross-domain knowledge understanding, for example, combining WeChat group chat records and shopping receipts to automatically calculate AA-style expense sharing.
  • Natural Language Processing: Step 3 performs excellently in natural language processing tasks and can understand and generate high-quality text content.