DiffusionGemma - Google's Open-Source Experimental Text Diffusion Model

What is DiffusionGemma

DiffusionGemma is an experimental open-source text diffusion model launched by Google DeepMind. The model is based on the Gemma 4 architecture and Gemini Diffusion research results, adopts a 26B MoE design, and by parallel denoising of 256-token text blocks, achieves a generation speed of 1000+ tokens/second on a single NVIDIA H100, about 4 times faster than traditional autoregressive models. The model supports bidirectional attention and real-time self-correction, and after quantization requires only 18GB VRAM to run locally on consumer-grade GPUs.

DiffusionGemma's Main Features

• Parallel Text Generation: Each forward pass simultaneously denoises 256 tokens, achieving parallel generation of entire blocks of text rather than word-by-word output.
• Bidirectional Context Reasoning: During generation, each token can attend to all other tokens in the same block, supporting nonlinear text tasks such as code infilling and inline editing.
• Real-Time Self-Correction: When the confidence of a token decreases, the sampler can re-noise it and replace and correct it in subsequent steps.
• Multimodal Input Processing: Supports interleaved input of text, images, and video, generates text output, with a context window of up to 256K tokens.
• Long-Sequence Block-Level Generation: Through a block autoregressive diffusion mechanism, after the already denoised 256-token block is committed to the KV cache, it continues generating the next block, balancing parallel speed and sequence stability.

DiffusionGemma's Technical Principles

• Uniform State Diffusion: DiffusionGemma's core mechanism draws on the ideas of image diffusion models: starting from a 256-token canvas filled with random placeholders, it gradually locks in high-confidence tokens through multiple denoising iterations, uses the locked tokens as contextual clues to resolve adjacent positions, and ultimately makes the entire sequence converge into coherent text. Each forward pass fixes about 15–20 tokens, greatly improving throughput through parallel processing.
• Shift in Hardware Bottleneck: Traditional autoregressive models are limited by memory bandwidth during single-user local inference, leaving GPU compute units largely idle. DiffusionGemma shifts the bottleneck from memory bandwidth to the compute side: by providing large-scale parallel denoising workloads to tensor cores, it keeps the GPU highly utilized in local inference scenarios, reaching 1000+ tokens/second on a single H100 and 700+ tokens/second on an RTX 5090.
• Hybrid Attention Architecture: The inference process alternates between two attention modes: the Prefill stage uses causal attention to process the input prompt and write it to the KV cache; the Denoising stage switches to bidirectional attention, allowing each token in the canvas to attend to all other tokens simultaneously. The architecture enables the model to handle constraint-intensive tasks.

How to Use DiffusionGemma

• Local Deployment with vLLM: Start the model with vLLM's OpenAI-compatible server, and after configuring the diffusion sampler and 256-token canvas parameters, it can provide inference services externally.
• Download Weights from Hugging Face: Visit the Hugging Face repository to obtain the model weight files open-sourced under the Apache 2.0 license.
• Run with Mainstream Inference Frameworks: The model natively supports direct loading and inference with frameworks such as Hugging Face Transformers, SGLang, and MLX.
• Hackable Diffusion Fine-Tuning: Use Google's open-source JAX research toolkit and official training recipes to conduct rapid experiments and fine-tuning for specific tasks.

Core Advantages of DiffusionGemma

• Extreme Local Inference Speed: In single-user local scenarios, it is about 4 times faster than autoregressive models of the same specification, achieving 1000+ tokens/second on H100 in real tests, significantly reducing latency for interactive development tools.
• Runs on Consumer-Grade Hardware: After quantization, it requires only 18GB VRAM, supports consumer-grade GPUs such as RTX 4090/5090 and DGX Spark desktop workstations, with no need for server-grade clusters.
• Zero-Day Open-Source Ecosystem Support: It is compatible at launch with mainstream toolchains such as vLLM, Hugging Face Transformers, MLX, and Unsloth, reducing integration costs.
• Expertise in Non-Linear Text Tasks: Bidirectional attention enables it to outperform traditional autoregressive models on tasks requiring global constraints, such as code infilling, structured output, amino acid sequences, and mathematical graphs.