LLaDA-Image - Ant Group's Open-Source Unified Image Generation and Editing Model

What is LLaDA-Image

LLaDA-Image is a 6B-parameter unifiedimage generationand editing model open-sourced by Ant Group's inclusionAI. The model adopts an innovative route of pure-image pretraining first, followed by language alignment, using a fully diffusion-based architecture (LLaDA2.0-mini for understanding + 6B DiT for generation) to achieve text-to-image, instruction editing, and Chinese-English text rendering. The Turbo version requires only 2–4 steps to produce an image, and ranks first among open-source models in both the Chinese and English tracks of Qwen-Image-Bench.

Main Features of LLaDA-Image

• Text-to-Image Generation: Generates high-fidelity, detail-rich photorealistic images based on natural language descriptions.
• Instruction-Based Image Editing: Upload a reference image and enter a natural language instruction to precisely modify specified regions while keeping the rest unchanged.
• Chinese-English Bilingual Text Rendering: Accurately renders Chinese and English text in generated images, supporting posters, typography, and artistic text design.
• VQ Conditional Generation: Uses visual quantization (VQ) tokens as conditional input to achieve controllable image generation.
• Reference Image Editing: Uses an uploaded image as a reference for editing operations such as style transfer and content modification.
• Turbo Fast Inference: The distilled model requires only 2–4 sampling steps to generate 1024×1024 high-quality images.

Technical Principles of LLaDA-Image

• Fully Diffusion-Based Unified Architecture: The model consists of an understanding module, LLaDA2.0-mini (an MoE-based diffusion language model), and a generation module, 6B DiT. Both use diffusion models as their backends and are bridged by a Connector, decoupling semantic understanding from pixel generation.
• Staged Training Strategy: Adopts a "learn to draw first, then learn to follow instructions" route: first conducting pure-image pretraining and mid-training to establish strong visual generation priors, then introducing paired language supervision to complete text-visual alignment.
• High-Quality Data Construction: Generation training cumulatively uses approximately 220 million samples, 98% of which are real images; image-text descriptions in the language alignment stage are generated and verified by Qwen series large models to ensure instruction-following accuracy.
• Efficient Training Optimization: The entire pipeline uses parameter-agnostic RMSNorm instead of LayerNorm and adopts the Muon optimizer, improving the stability and efficiency of large-scale training.
• Twin-DMD Fast Distillation: The Turbo version compresses the Base model through Twin-DMD distillation technology, generating 1024×1024 high-quality images with 2–4 sampling steps, achieving a balance between speed and quality.

Follow WeChat and reply "Open Source" to join theAI Open-Source Project Communication Group

How to Use LLaDA-Image

• Environment Setup: Install dependencies such as Python 3.11, PyTorch 2.8, Diffusers 0.39.0, and FlashAttention 2.8.3 to set up a local inference environment.
• Obtain Weights: Download the model files for LLaDA-Image (high-fidelity version) or LLaDA-Image-Turbo (fast version) from Hugging Face or ModelScope.
• Text-to-Image Generation: After loading the model, enter the description text. For the Base version, set 50 steps and guidance scale 5.0; for the Turbo version, set 2–4 steps and guidance scale 1.0, to generate 1024×1024 images.
• Instruction-based image editing: Upload a reference image and enter a natural language editing instruction (e.g., "change the background to a seaside"), and the model automatically keeps unedited areas unchanged.
• Note the size specifications: The input size for text-to-image and VQ conditional generation must be a multiple of 16, and for editing tasks must be a multiple of 32.