Qwen3 - Alibaba Tongyi's new generation open-source hybrid reasoning model series

What is Qwen3

Qwen3 is a new generation large language model launched by Alibaba. Qwen3 supports two working modes: "thinking mode" and "non-thinking mode." In thinking mode, the model reasons step by step and gives a final answer after careful consideration, making it suitable for complex problems. In non-thinking mode, the model provides fast, near-instant responses, making it suitable for simple problems. Qwen3 supports 119 languages and dialects, a significant improvement in language capability compared with the 29 languages of the previous generation. Qwen3 optimizes coding and Agent capabilities and supports the MCP protocol, enabling better integration with external tools and data sources. Qwen3's dataset scale reaches about 36 trillion tokens, twice that of Qwen2.5. It adopts a four-stage training process, including long chain-of-thought cold start, long chain-of-thought reinforcement learning, thinking mode fusion, and general reinforcement learning. The Qwen3 series models are open-sourced under the Apache 2.0 license, and developers, research institutions, and enterprises worldwide can download and use them commercially for free.

Alibaba's latest two open-source Qwen3 series models, Qwen3-Embedding and Qwen3-Reranker.

Qwen3-Embedding: In multilingual text representation benchmarks, Qwen3-Embedding's performance is outstanding. Among them, the 8B parameter-scale model ranks first on the MTEB Multilingual Leaderboard with a high score of 70.58, surpassing many commercial API services, such as Google's Gemini-Embedding. It takes a single piece of text as input and takes the hidden state vector corresponding to the model's last-layer "EOS" token as the semantic representation of the input text. It is suitable for scenarios that require semantic representation of text, such as text classification, clustering, and similarity calculation, and can provide high-quality text features for downstream tasks.

Qwen3-Reranker: In basic relevance retrieval tasks, the 8B model achieved a high score of 69.02 in multilingual retrieval tasks, a score of 77.45 in Chinese retrieval tasks, and a score of 69.76 in English retrieval tasks, significantly outperforming other baseline models. It takes text pairs (such as a user query and a candidate document) as input and uses a single-tower structure to calculate and output the relevance score of the two texts. It is used for text retrieval and ranking tasks, such as ranking results in search engines and ranking answers in question-answering systems, and can improve the relevance and accuracy of search results.

Qwen3's main features

• Hybrid reasoning mode: Qwen3 supports two working modes: "thinking mode" and "non-thinking mode."Thinking modeis suitable for complex problems, and the model reasons step by step before giving an answer;Non-thinking modeprovides fast, near-instant responses and is suitable for simple problems. It allows users to flexibly control the model's reasoning process according to the complexity of the task, achieving a balance between cost efficiency and reasoning quality.
• Multilingual support: Qwen3 supports 119 languages and dialects, including English, French, Chinese (Simplified and Traditional), Cantonese, and more, greatly expanding its international application scope.
• Enhanced Agent capabilities: Qwen3 optimizes coding and Agent capabilities, supports the MCP protocol, and can interact efficiently with external tools. Combined with the Qwen-Agent framework, it can greatly reduce coding complexity and enable efficient tasks such as mobile phone and computer Agent operations.
• Multiple model configurations: Qwen3 provides multiple model configurations, including two MoE models (Qwen3-235B-A22B and Qwen3-30B-A3B) and six Dense models (Qwen3-32B, Qwen3-14B, Qwen3-8B, Qwen3-4B, Qwen3-1.7B, and Qwen3-0.6B), covering various scenarios from small devices to large-scale enterprise deployments.

Technical Principles of Qwen3

• Large-Scale Pretraining: Qwen3's pretraining data volume reaches approximately 36 trillion tokens, twice that of Qwen2.5, covering 119 languages and dialects. The pretraining process is divided into three stages:
  • First Stage (S1): The model was pretrained on over 30 trillion tokens with a context length of 4K tokens. This stage provided the model with basic language skills and general knowledge.
  • Second Stage (S2): The dataset was improved by increasing the proportion of knowledge-intensive data (such as STEM, programming, and reasoning tasks), after which the model was further pretrained on an additional 5 trillion tokens.
  • Third Stage: High-quality long-context data was used to extend the context length to 32K tokens, ensuring the model can effectively handle longer inputs.
• Optimized Post-Training: To develop a hybrid model capable of both thinking reasoning and rapid response, Qwen3 implemented a four-stage training pipeline:
  • Long Chain-of-Thought Cold Start: The model was fine-tuned using diverse long chain-of-thought data, covering various tasks and domains such as mathematics, code, logical reasoning, and STEM problems.
  • Reasoning-based Reinforcement Learning: Rule-based rewards were used to enhance the model's exploration and deep-diving capabilities.
  • Thinking Mode Fusion: The model was fine-tuned on a combination of long chain-of-thought data and commonly used instruction fine-tuning data, integrating the non-thinking mode into the thinking model.
  • General Reinforcement Learning: Reinforcement learning was applied to tasks in over 20 general domains, including instruction following, format following, and Agent capabilities, to further enhance the model's general capabilities and correct undesirable behaviors.
• Multiple Model Configurations: Qwen3 offers multiple model configurations, including:
  • MoE Models: Qwen3-235B-A22B and Qwen3-30B-A3B.
  • Dense Models: Qwen3-32B, Qwen3-14B, Qwen3-8B, Qwen3-4B, Qwen3-1.7B, and Qwen3-0.6B. Covering various scenarios from small devices to large-scale enterprise deployments.
• Performance Optimization: Qwen3's performance is significantly improved while deployment costs are greatly reduced. For example, only 4 H20 GPUs are needed to deploy the full version, and the VRAM usage is only one-third that of models with similar performance.

Qwen3 Project Address

• Project Official Website：https://qwenlm.github.io/blog/qwen3/
• Github Repository：
  • Qwen3：https://github.com/QwenLM/Qwen3
  • Qwen3-Embedding：https://github.com/QwenLM/Qwen3-Embedding
• HuggingFace Model Library：
  • Qwen3：https://huggingface.co/collections/Qwen/qwen3
  • Qwen3-Embedding：https://huggingface.co/collections/Qwen/qwen3-embedding
  • Qwen3-Reranker：https://huggingface.co/collections/Qwen/qwen3-reranker

Qwen3 Performance Results

Qwen3 performs excellently in multiple benchmark tests, for example:

• AIME25: Qwen3 scored 81.5 points, setting a new open-source record.
• LiveCodeBench: Qwen3 scored over 70 points, performing even better than Grok3.
• ArenaHard: Qwen3 surpassed OpenAI-o1 and DeepSeek-R1 with 95.6 points.

Qwen3 Application Scenarios

• Text Generation: Qwen3 can generate coherent and natural long texts, suitable for tasks such as automated writing, news generation, and blog article creation. It can generate complete articles or stories based on given prompts.
• Machine Translation: Qwen3 supports 119 languages and dialects, and performs excellently in multilingual translation tasks. It can handle translation tasks between multiple language pairs and provide high-quality translation results.
• Automatic generation of legal documents: Qwen3 can generate legal documents such as contracts, legal opinions, and litigation documents. By fine-tuning on corpora in the legal domain, Qwen3 can generate formatted documents that comply with legal regulations.
• Technical documentation writing: Qwen3 can generate detailed technical documentation, product descriptions, user manuals, and more. By fine-tuning on corpora in the technical domain, Qwen3 can help developers and technical support teams automatically generate documentation that meets industry standards.
• Medical field: Qwen3 can be used to generate medical reports, diagnostic recommendations, and more. By fine-tuning on data such as medical literature and medical records, Qwen3 can generate professional reports that meet medical standards. It can assist doctors in automatically generating case records during the diagnostic process.
• Legal field: By fine-tuning on legal literature, precedents, and regulations, Qwen3 can generate highly specialized legal documents.