What Is Mellum2
Mellum2 is JetBrains' new open-source machine learning model for software engineering systems. The model activates only 2.5B parameters per token and is specifically optimized for software engineering AI workflows, supporting code generation, intelligent routing, sub-agent invocation, and private deployment. With low latency, high throughput, and low cost, Mellum2 leads in code evaluations such as LiveCodeBench, making it a high-performance choice for building enterprise-grade AI programming assistants and automated workflows.
Mellum2's Main Features
-
Intelligent Code Generation: Trained on code and natural language data, supporting code completion, function generation, and refactoring suggestions.
-
Dual-Mode Reasoning: Offers two modes, Thinking (deep reasoning) and Non-thinking (fast response), switchable on demand.
-
Workflow Routing and Summarization: Acts as an intelligent router for AI workflows, responsible for task distribution, document summarization, and intermediate reasoning.
-
Tool Calling and Agent Collaboration: Supports sub-agent invocation and toolchain integration, and can be embedded into complex automated processes.
-
Local Private Deployment: Can run independently on enterprise intranets or local devices without relying on external APIs.
Mellum2's Technical Principles
- MoE Sparse Architecture: Adopts a Mixture-of-Experts design with 64 experts and 8 experts activated per token, with 12B total parameters and only 2.5B actually activated, bringing inference cost close to that of a 2.5B dense model. Combined with Grouped-Query Attention and sliding window attention, it significantly compresses memory usage and accelerates decoding. It introduces a Multi-Token Prediction head as an auxiliary pretraining objective to improve performance, serving as a built-in Draft model for speculative decoding to further reduce latency.
- Three-Stage Curriculum Pretraining: Trained on approximately 10.6 trillion tokens, with the data mix gradually transitioning from general web data to curated code and mathematical content, forming a progressive curriculum. Using the Muon optimizer with FP8 mixed precision and a Warmup-Hold-Decay (linear decay to zero) learning rate schedule, it strikes a balance between training efficiency and model stability.
- Long-Context Extension: The pretrained base model extends its context window to 128K through Layer-Selective YaRN, applying positional encoding interpolation only to key layers and avoiding the performance loss caused by full-layer fine-tuning.
Mellum2's Core Advantages
-
Efficient MoE Architecture: 12B total parameters with only 2.5B activated, significantly reducing inference costs and achieving high throughput and low latency
-
Vertical Domain Specialization: Abandons multimodality and focuses on code and natural language, making it more precise and lightweight in software engineering scenarios
-
Production-Grade Performance: In the LiveCodeBench v6 code evaluation, the Thinking mode scores 69.9, leading open-source models of the same tier
-
Fully Open Source and Commercially Usable: Under the Apache 2.0 license, free to experiment with, fine-tune, and deploy for large-scale commercial use
-
Enterprise Privacy Friendly: Supports fully localized operation, meeting code security and data compliance requirements
Comparison of Mellum2 with Similar Competing Products
| Comparison Dimension | Mellum2 | Qwen3.5-9B | SeedCoder-8B |
|---|---|---|---|
| Model Architecture | 12B MoE (64 experts, 8 activated, 2.5B active parameters) | 9B Dense Model (Dense) | 8B Dense Model (Dense) |
| Open Source License | Apache 2.0(Fully commercially usable) | Open source (commercially usable) | Unclear/Partially restricted |
| Modality Support | Text + code only (vertical specialization) | Text, code, images, video (general-purpose multimodal) | Code only (single domain) |
| Compute per Token | ≈2.5B parameters(extremely low) | 9B parameters (fully activated) | 8B parameters (fully activated) |
| LiveCodeBench v6 | 69.9（Thinking） | 68.3（Thinking） | 28.1（Non-thinking） |
| BFCL V4 Tool Calling | 45.6（Thinking） | 42.7（Thinking） | N/A (not supported) |
| AIME Mathematical Reasoning | 58.4（Thinking） | 73.4（Thinking） | 0 (not supported) |
| Context Length | 128K (YaRN extension) | 128K+ | Typically 4K-8K |
| Reasoning Mode | Dual mode: Thinking + Non-thinking | Dual mode: Thinking + Non-thinking | Non-thinking only |
Application Scenarios of Mellum2
-
Intelligent Code Completion and Generation: Provides real-time code completion, function generation, code refactoring, and editing suggestions in the IDE, replacing traditional autocomplete tools.
-
Debugging and Error Diagnosis: Assists developers in locating bugs, analyzing stack traces, and generating fix solutions, reducing the time cost of debugging.
-
Multi-step Reasoning and Complex Task Decomposition: Serves as an intermediate reasoning node in Agent workflows, decomposing complex requirements into executable sequences of subtasks.
-
Tool Calling and Function Orchestration: Connects to external APIs, databases, or development tools through Function Calling to enable automated build, test, and deployment processes.
-
Conversational Programming Assistant: Answers technical questions, explains code logic, and recommends best practices through natural language interaction, acting as a 24/7 technical advisor.