Qwen3.8-27B-DFlash2 - Inco AI's Open-Source Speculative Decoding Draft Model

What is Qwen3.8-27B-DFlash2

Qwen3.8-27B-DFlash2 is an open-source speculative decoding draft model from Inco AI, designed specifically for Qwen3.8-27B . The model has only 1.92B parameters and, through block-level parallel prediction, a lightweight path selector, and dual-tap dynamic convolution, generates an entire block of token drafts in parallel in a single forward pass. On a single H200 GPU with SGLang at single concurrency, throughput reaches 2.7–3.4 times that of autoregressive decoding, with an average acceptance length of 4.80, outperforming native MTP and the community DSpark, while decoding results remain lossless.

Main Features of Qwen3.8-27B-DFlash2

• Parallel Draft Generation: As a speculative decoding draft model for Qwen3.8-27B, it can predict an entire block of tokens in parallel in a single forward pass, replacing traditional token-by-token autoregressive draft generation.
• Intelligent Path Selection: Through a lightweight path selector, it traces a coherent path among the top-16 candidates at each position, resolving the unnatural token transitions caused by parallel prediction.
• Local Dependency Modeling: It uses dual-tap dynamic convolution to specifically model local dependencies between adjacent positions within a block, significantly alleviating the "suffix decay" phenomenon at block-tail positions.
• Lossless Inference Acceleration: On mainstream engines such as SGLang, vLLM, and llama.cpp, it achieves single-concurrency throughput 2.7–3.4 times that of autoregressive decoding, while strictly guaranteeing lossless decoding output.

Technical Principles of Qwen3.8-27B-DFlash2

• Block-Level Parallel Draft Generation: The draft model of traditional speculative decoding still needs to generate token by token autoregressively. DFlash changes the drafting process to non-autoregressive as well, predicting in parallel the tokens at all positions within an entire block through a single forward pass.
• Lightweight Path Selector: Since the top-1 candidates independently predicted at each position are not necessarily coherent with one another, the selector retains the top-16 candidates at each position, scores all adjacent candidate pairs in parallel (based on low-rank bilinear attention and context gating), and traces the best successor path starting from the last verified token via greedy or sampling, increasing the acceptance length by about 0.4 tokens with only 2M additional parameters.
• Dual-Tap Dynamic Convolution: Observing that deep attention's investment in intra-block dependencies continuously decays, the team inserts content-adaptive dual-tap dynamic depthwise convolution before and after the Attention and MLP sublayers. Each position mixes its own representation with the preceding representation, taking on intra-block local modeling at extremely low cost (+3% parameters, +0.7% latency), allowing attention to return to context reading, with an effect close to adding ten Transformer layers.

Follow WeChat and reply "open source" to join theAI Open-Source Project Exchange Group

How to Use Qwen3.8-27B-DFlash2

• Model Preparation: Download the Qwen3.8-27B target model and the Qwen3.8-27B-DFlash2 draft model locally via ModelScope.
• SGLang Deployment: After installing sglang, run sglang.launch_server, specifying --speculative-algorithm DFLASH and the draft model path to start the accelerated service.
• vLLM Deployment: After installing the corresponding version of vLLM, use vllm serve, setting method to --speculative-config in dflash and filling in the draft model name.
• llama.cpp Deployment: Pull and compile the PR branch that supports DFlash, load the GGUF files of the target model and the draft model respectively, and use --spec-type draft-dflash to start the server.
• Apple Silicon: Using the oMLX precompiled package, enable DFlash for the target model in Model Manager, specify the draft model, and set Verify mode to dflash.
• Command-line tool: After installing the dflash package, you can directly use the dflash generate command together with transformers, MLX, or an OpenAI-compatible backend to run inference and evaluation.

Core advantages of Qwen3.8-27B-DFlash2

• Extremely small size: A draft model with only 1.92B parameters (about 3.85GB) can provide significant acceleration for the 27B target model, with extremely low deployment cost.
• Parallel drafting: It breaks through the limitations of traditional autoregressive drafting, generating an entire block of tokens in parallel in a single forward pass, greatly reducing latency in the drafting stage.
• Lossless acceleration: On a single H200 GPU, it achieves 2.7–3.4 times the throughput of autoregressive decoding at single concurrency, while strict verification ensures the output is completely lossless.
• Precise selection: The lightweight path selector uses only 2M parameters and 0.6% latency overhead to track coherent paths from the top-16 candidates, significantly improving the draft acceptance rate.
• Local modeling: Dual-tap dynamic convolution uses 3% of the parameters to specifically model intra-block dependencies, effectively alleviating the "suffix decay" at deep positions, with an effect comparable to adding ten Transformer layers.
• Ecosystem compatibility: It has been natively integrated with mainstream inference engines such as SGLang, vLLM, llama.cpp, and oMLX, supporting multi-platform deployment on NVIDIA and Apple Silicon.

Project addresses of Qwen3.8-27B-DFlash2

• Project website：https://inco.ai/blog/dflash2/
• GitHub repository：https://github.com/z-lab/dflash

Comparison of Qwen3.8-27B-DFlash2 with similar competitors

Application scenarios of Qwen3.8-27B-DFlash2

• AI Agent services: It greatly reduces the per-token decoding cost during long-running Agents, supporting high-throughput execution flows such as tool calling and task planning.
• Real-time dialogue systems: The 2.7–3.4 times acceleration at single concurrency significantly reduces response latency, improving the interaction experience of chatbots and customer service assistants.
• Code generation assistants: It leads in acceptance length on the HumanEval and MBPP benchmarks, making it suitable for scenarios requiring fast completion such as IDE plugins and programming Copilots.
• Long-form text creation: The high acceptance length allows more tokens to be produced per verification, accelerating the generation of long documents such as articles, reports, and marketing copy.
• Local deployment on edge devices: The small 1.92B size, together with oMLX and llama.cpp support, enables efficient local inference on devices such as Apple Silicon laptops.