ERNIE-Image - Baidu Wenxin's Open-Source Text-to-Image Model

What is ERNIE-Image

ERNIE-Image is an 8B-parameter open-sourcetext-to-imagemodel from Baidu's Wenxin team, based on the Diffusion Transformer architecture, focusing on high controllability and precise long-text rendering. The model can accurately generate complex visual content such as bilingual Chinese-English posters, comics, and infographics, solving the problem of text blur and hallucination. ERNIE-Image offers dual versions: a Standard version (50-step high quality) and a Turbo version (8-step fast), requiring only 24GB of VRAM to run locally, providing designers and developers with an out-of-the-box image generation solution for Chinese scenarios.

Main Features of ERNIE-Image

• Highly Controllable Generation: The model supports complex structured layout control and can precisely arrange the spatial positional relationships between multiple objects, achieving a high score of 0.8856 on the GENEval benchmark.
• Long-Text Rendering: Specially optimized for the precise presentation of long bilingual Chinese-English text in images, scoring 0.9733 on the LongTextBench test, particularly suitable for poster and comic creation.
• Dual-Version Mode: Offers two inference modes: the Standard version (50-step high-quality rendering) and the Turbo version (8-step fast distillation), respectively meeting the needs for refined creation and rapid iteration.
• Full-Process Handling: Built-in complete image processing workflow covering four stages: generation, editing, compositing, and upscaling, supporting end-to-end creation from sketch to finished product.
• Intelligent Prompt Enhancement: Equipped with a lightweight Prompt Enhancer module that can automatically expand short user input into structurally rich and detailed generation descriptions.

Technical Principles of ERNIE-Image

• Single-Stream DiT Architecture: Adopts a Diffusion Transformer to uniformly encode text and image signals, achieving cross-modal deep alignment at an 8B parameter scale, improving generation consistency and complex scene understanding.
• Chinese Typography Optimization: Specially trained for Chinese character structure and typography logic, improving attention mechanisms and positional encoding, solving the problems of text hallucination, blur, and structural errors in traditional diffusion models.
• Distillation Acceleration Technology: The Turbo version transfers the capabilities of the teacher model to an 8-step lightweight student model through knowledge distillation, significantly improving inference speed while maintaining usable quality.
• Consumer-Grade Hardware Adaptation: Requires only 24GB of VRAM to run locally, natively supports 1024×1024 resolution, lowering the deployment threshold.

How to Use ERNIE-Image

• Environment Preparation: Ensure the local environment is equipped with an NVIDIA GPU with 24GB or more VRAM, install Python 3.8+ and configure transformers>=4.50.0、torch、diffusers and other dependency libraries.
• Load the Standard Model: Via DiffusionPipeline.from_pretrained("Baidu/ERNIE-Image", torch_dtype=torch.float16, trust_remote_code=True) load the 8B parameter model and move it to the CUDA device, suitable for 50-step high-quality generation scenarios.
• Load the Turbo Model: Via DiffusionPipeline.from_pretrained("Baidu/ERNIE-Image-Turbo", torch_dtype=torch.float16, trust_remote_code=True) load the distilled model, supporting 8-step fast inference to meet draft iteration needs.
• Execute Basic Text-to-Image: Call pipe(prompt="描述文本", num_inference_steps=50, guidance_scale=7.5, height=1024, width=1024) passing in the text prompt and parameters, then call .images[0] on the returned image object to get the result and save it.
• Use Intelligent Prompt Enhancement: The built-in Prompt Enhancer automatically expands short input into structured descriptions, improving generation quality without the need to manually write complex prompts.
• Call the image editing feature: Use the pipe.edit(prompt="修改指令", image=原图, mask=遮罩) API to perform local inpainting on specified regions, enabling precise mask-based content modification.
• Enable low-VRAM mode: When VRAM is less than 24GB, add device_map="auto" or call pipe.enable_sequential_cpu_offload() to implement layer offloading, completing inference on consumer-grade GPUs.

Key information and usage requirements of ERNIE-Image

• Basic specifications: Based on an 8B-parameter single-stream Diffusion Transformer architecture, supporting 1024×1024 resolution image generation.
• Dual-version design: The standard version is a 50-step SFT model for final high-quality rendering; the Turbo version is an 8-step distilled model for rapid draft iteration.
• Core capabilities: Precise Chinese-English bilingual long-text rendering (LongTextBench score 0.9733), highly controllable layout generation (GENEval score 0.8856), supporting a four-stage workflow of generation/editing/synthesis/upscaling.
• Open-source license: Uses the Apache-2.0 license, supports commercial applications and secondary development, and can be deployed locally on consumer-grade GPUs (24GB VRAM).
• Hardware configuration: Requires an NVIDIA GPU with 24GB or more VRAM (such as RTX 4090) to meet the inference requirements of the 8B-parameter model.
• Software environment: Requires Python 3.8 or higher, and installation of transformers>=4.50.0、torch、diffusers and other core dependency libraries.

Core advantages of ERNIE-Image

• Precise Chinese long-text rendering: Specially optimized for the accurate presentation of Chinese-English bilingual long text in images, with a LongTextBench test score of 0.9733, thoroughly solving problems such as text blurring and structural hallucinations in traditional diffusion models, and is particularly suitable for creative scenarios requiring precise typesetting such as posters, comics, and infographics.
• Highly controllable structured generation: Supports precise control of complex page layouts and spatial relationships among multiple objects, with a GENEval benchmark score of 0.8856, able to accurately arrange element positions according to structured descriptions and achieve multi-object compositional generation with strict instruction adherence.
• Flexible dual-version adaptation: Provides a standard version (50-step SFT) for final high-quality rendering, and a Turbo version (8-step distillation) for rapid draft iteration, allowing users to flexibly choose the inference mode according to quality or speed requirements.
• Deployable on consumer-grade hardware: Only 24GB VRAM is needed to run the 8B-parameter model locally on a single consumer-grade GPU (such as RTX 4090), and optimization solutions such as layer offloading are provided, significantly lowering the threshold for enterprises to obtain text-to-image capabilities.

Project address of ERNIE-Image

• Project official website：https://ernie.baidu.com/blog/posts/ernie-image/
• HuggingFace model repository：
  • https://huggingface.co/baidu/ERNIE-Image
  • https://huggingface.co/Baidu/ERNIE-Image-Turbo

Comparison of ERNIE-Image with similar competing products

Application scenarios of ERNIE-Image

• Commercial poster design: Using precise long-text rendering capabilities, generate Chinese and English posters containing brand slogans, promotional information, and product specifications. Suitable for scenarios such as e-commerce promotions, event publicity, and outdoor advertising, with text that is clear and readable without post-editing.
• Comic and illustration creation: Supports complex panel layouts and multi-character scene control, and can generate comic pages with speech bubbles, narration text, and onomatopoeia, enabling one-stop production from draft to finished artwork.
• Infographics and data visualization: Converts structured data descriptions into visual infographics containing charts, labels, and explanatory text. Suitable for creating content such as report covers, teaching materials, and long-form social media graphics.
• E-commerce product images: High controllability supports precise arrangement of the spatial positions of product subjects, background elements, price tags, and selling-point copy, quickly generating product main images and detail pages that meet platform specifications.
• Publishing and print materials: Precise Chinese character rendering ensures that text in printed materials such as book covers, magazine inner pages, and brochures is accurate, avoiding the garbled text problems of traditional AI-generated images.