SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

ERNIE-4.5-21B-A3B-Thinking - A Thinking Model Launched by Baidu

Author: AI小集 · Original source: AI工具集

What is ERNIE-4.5-21B-A3B-Thinking

ERNIE-4.5-21B-A3B-Thinking is a large language model launched by Baidu that focuses on reasoning tasks. It adopts a Mixture of Experts (MoE) architecture, with a total of 21 billion parameters, activating 3 billion parameters per token, and supports a 128K long context window, making it suitable for complex reasoning tasks. The model builds its language core backbone through text pretraining. In the reasoning-enhanced post-training stage, it utilizes techniques such as supervised fine-tuning (SFT) and progressive reinforcement learning (PRL) to significantly improve capabilities in logical reasoning, mathematical computation, scientific question answering, and more. It supports efficient tool calling and can be integrated with vLLM, Transformers 4.54+, and FastDeploy, making it suitable for scenarios such as program synthesis, symbolic reasoning, and multi-agent workflows.

ERNIE-4.5-21B-A3B-Thinking

Main Features of ERNIE-4.5-21B-A3B-Thinking

  • Powerful Reasoning Ability: ERNIE-4.5-21B-A3B-Thinking performs excellently in fields requiring reasoning ability, such as logical reasoning, mathematical computation, and scientific question answering. It can handle complex reasoning tasks and provide users with accurate answers.
  • Efficient Tool Calling: The model supports structured tool and function calling and can be integrated with vLLM, Transformers 4.54+, and FastDeploy to achieve more efficient task execution and feature expansion.
  • Long Context Understanding: It has a 128K context window and can understand and process long text information, making it suitable for complex reasoning tasks that require long context, such as long document analysis and multi-step reasoning.
  • Multi-domain Applications: It is widely used in scenarios such as program synthesis, symbolic reasoning, and multi-agent workflows, providing solutions for complex tasks in different domains and meeting diverse business needs.
  • Open Source and Easy to Use: It is open-sourced under the Apache-2.0 license and can be used on platforms such as Hugging Face, making it convenient for developers to conduct research and commercial deployment and lowering the barrier to use.

Technical Principles of ERNIE-4.5-21B-A3B-Thinking

  • Mixture of Experts Architecture: ERNIE-4.5-21B-A3B-Thinking adopts a Mixture of Experts (MoE) architecture, dividing model parameters into multiple expert modules. Each input token activates only part of the expert modules, significantly improving computational efficiency while maintaining model performance.
  • Long Context Window: The model supports a 128K context window and can process long text input, which is crucial for tasks requiring long context understanding, such as complex reasoning and long document analysis.
  • Reasoning-Enhanced Training: Through techniques such as supervised fine-tuning (SFT) and progressive reinforcement learning (PRL), the model undergoes specialized training for reasoning ability, enabling it to perform excellently in tasks such as logical reasoning, mathematical computation, and scientific question answering.
  • Activation Mechanism: The model is designed with an efficient activation mechanism, with each token activating 3B parameters, ensuring efficient operation even with large-scale parameters while maintaining the model's flexibility and adaptability.

Project Address of ERNIE-4.5-21B-A3B-Thinking

  • HuggingFace Model Library：https://huggingface.co/baidu/ERNIE-4.5-21B-A3B-Thinking

Application Scenarios of ERNIE-4.5-21B-A3B-Thinking

  • Complex Reasoning Tasks: It is suitable for scenarios requiring deep thinking and reasoning, such as logical reasoning, mathematical computation, and scientific question answering, providing accurate analysis and solutions.
  • Code Generation and Optimization: It can generate and optimize code, helping developers improve programming efficiency, and is suitable for tasks such as program synthesis and symbolic reasoning.
  • Multi-agent workflows: It supports multi-agent collaboration and can be used to build complex automated workflows, improving task execution efficiency.
  • Long-text analysis: With a long context window, it can handle tasks such as long-document analysis and complex text reasoning, and is suitable for scenarios such as academic research and business report analysis.
  • Tool calling and integration: It supports structured tool and function calling, can integrate with a variety of platforms and tools, expands application scenarios, and meets diverse business needs.