DSpark - DeepSeek's open-source speculative decoding acceleration framework in collaboration with Peking University

What is DSpark

DSpark is DeepSeek an open-source speculative decoding acceleration framework in collaboration with Peking University, specifically addressing the pain point of slow autoregressive generation in large models, which is like "squeezing toothpaste." It adopts a semi-autoregressive generation architecture, modeling inter-token dependencies through a lightweight Markov head, balancing the speed and coherence of parallel drafting; at the same time, it introduces confidence-scheduled verification, dynamically allocating verification resources based on system load. The framework has been deployed in the DeepSeek-V4-Flash/Pro production environment, improving single-user generation speed by 57%–85% and increasing throughput by up to 400%. The project is open-sourced under the MIT license, compatible with mainstream models such as Qwen and Gemma, providing a practical engineering solution for efficient inference of large models.

DSpark's main features

Semi-autoregressive draft generation: Retains the high-speed characteristics of parallel draft models while adding a lightweight Markov head (or RNN head) to model adjacent token dependencies, alleviating the suffix decay problem of traditional parallel approaches and making candidate sequences more coherent from front to back.
Confidence score prediction: Outputs a confidence score for each candidate token, estimating in real time the probability that the token will be accepted by the target model, providing a decision basis for subsequent scheduling.
Hardware-aware prefix scheduling: Dynamically determines the token length to be verified for each request based on system concurrency load, candidate confidence, and engine throughput curves—verifying more when the system is idle and streamlining low-confidence requests when busy.
Production-grade inference acceleration: Has been integrated into DeepSeek-V4-Flash/Pro online services, achieving a 57%–85% improvement in single-user generation speed and up to a 400% increase in aggregate throughput under real high-concurrency traffic.
Multi-model compatibility support: In addition to DeepSeek's own models, it is compatible with mainstream open-source large models such as Qwen (Tongyi Qianwen) and Gemma.
Full-stack open source: Open-sources the complete code, paper, training scripts, and model checkpoints under the MIT license, lowering the barrier for developers to get started.

How to use DSpark

Clone the project: Clone the DeepSpec open-source repository, configure the runtime environment, and install the relevant dependencies
Download models: Download the target model (such as DeepSeek-V4, Qwen3, or Gemma4) and the corresponding DSpark draft model checkpoints
Select dependency modules as needed: Load the semi-autoregressive draft model and select either the Markov head or the RNN head as the sequential dependency module as needed
Scheduled verification: Enable confidence-scheduled verification and configure the hardware-aware prefix scheduler to adapt to the concurrent load and throughput curves of the current GPU cluster
Integrate with the engine: Integrate DSpark into an existing inference engine (such as vLLM or a custom service framework), replacing traditional MTP-1 or standard autoregressive decoding processes
Send requests:Send requests via API or command line, and the system automatically executes the accelerated inference process of "draft generation → confidence evaluation → dynamic verification → return results"

DSpark's official website address

Technical paper：https://github.com/deepseek-ai/DeepSpec/blob/main/DSpark_paper.pdf

DSpark's core advantages

Semi-autoregressive architecture, balancing speed and coherence: It retains the high-throughput advantage of parallel draft models while modeling inter-token dependencies through a lightweight Markov head (or RNN head), effectively alleviating the suffix decay problem of traditional parallel approaches, making drafts more coherent before and after and achieving a higher acceptance rate.
Confidence-based dynamic scheduling, more intelligent resource utilization: It introduces confidence score prediction and a hardware-aware prefix scheduler, dynamically adjusting verification length based on system concurrency load, candidate survival probability, and engine throughput curves. When the system is idle, it verifies more; when busy, it streamlines low-confidence requests to avoid wasting batch capacity.
Significant production-grade performance improvement: It has been deployed in DeepSeek-V4-Flash/Pro online services, achieving per-user generation speed improvement of 57%–85%in real high-concurrency traffic, with aggregate throughput improving by up to 400%。
Broad model compatibility: It supports not only DeepSeek's own models but also mainstream open-source large models such as Qwen (Tongyi Qianwen) and Gemma, offering flexible application scenarios.
Full-stack open source, low integration barrier: The complete code, paper, training scripts, and model checkpoints are open-sourced under the MIT license, allowing developers to quickly integrate them into vLLM or custom inference engines.
Zero quality loss acceleration: Based on the speculative decoding mechanism, the target model output distribution remains unchanged, significantly increasing speed without sacrificing the accuracy or quality of generated content.

Comparison of DSpark with similar competitors

Comparison dimension	DSpark（DeepSeek）	Eagle3(representative of autoregressive drafting)	DFlash(representative of parallel drafting)
Technical approach	Semi-autoregressive generation + confidence-scheduled verification	Pure autoregressive draft model	Pure parallel draft model
Draft generation method	Fast generation of parallel blocks + Markov/RNN head to model intra-block dependencies	Token-by-token sequential draft generation	One-shot parallel generation of an entire candidate block
Dependency modeling capability	Strong: Explicitly models adjacent token transition relationships through a lightweight sequential module	Strong: Naturally preserves complete autoregressive dependencies, with coherent context	Weak: Tokens within a block lack sequential dependencies and are prone to inconsistent combinations
Verification strategy	Dynamic scheduling: real-time adjustment of verification length based on confidence scores and system load	Fixed or heuristic verification length	Usually verifies the entire candidate block at a fixed length
Speed vs consistency	Balanced: The parallel backbone ensures speed, while the sequential module alleviates suffix decay	High consistency but the drafting stage itself is slower; the longer the candidate, the greater the disadvantage	Fast but with severe suffix decay; the further along, the lower the acceptance rate
Production environment adaptation	Hardware-aware prefix scheduler, dynamically allocating batch capacity based on concurrency load	Requires additional optimization to adapt to high-concurrency scheduling	Easily wastes batch capacity verifying low-confidence tokens
Typical performance	Average acceptance length improved compared with Eagle3 by 26.7%–30.9%; compared with DFlash, improved by 16.3%–18.4%	Medium acceptance length, performs better on short sequences	The acceptance length is initially high but decays rapidly, so efficiency drops on long sequences.

Application Scenarios of DSpark

Real-time chat and dialogue systems: For low-latency requirements in highly interactive scenarios, DSpark can significantly increase single-user generation speed and improve dialogue fluency and user experience.
Code assistants and programming tools: Code generation tasks have a high candidate token acceptance rate (average accepted length reaches 5.12), and DSpark can accelerate code completion, automatic error correction, and multi-file generation.
Multi-turn Agent workflows: In complex tasks involving multi-turn calls and chained toolchains, it reduces the response latency of each turn and prevents latency from compounding and amplifying with the number of turns.
Mathematical reasoning and online education: Mathematical tasks (GSM8K, MATH, AIME, etc.) have the highest candidate acceptance rate (average 5.57), making it suitable for problem-solving scenarios with long reasoning steps and strong structure.
High-concurrency cloud API services: Through a hardware-aware prefix scheduler that dynamically adapts to GPU load, it achieves up to a 400% increase in throughput under real high-concurrency traffic and reduces the cost per request.
Local deployment of open-source models: It is compatible with mainstream models such as Qwen and Gemma and is open-sourced under the MIT license, making it easy for small and medium-sized enterprises and developers to integrate efficient inference capabilities into self-developed frameworks or engines such as vLLM.