Seedream 2.0 - ByteDance Doubao's Native Chinese-English Bilingual Image Generation Model

What is Seedream 2.0

Seedream 2.0 is a native Chinese-English bilingual image generation model launched by the ByteDanceDoubaolarge model team, addressing the shortcomings of existing models in text rendering and cultural understanding. The model uses a self-developed bilingual large language model (LLM) as a text encoder, capable of directly learning local knowledge from massive data to generate high-fidelity images with accurate cultural details and aesthetic expression. Seedream 2.0 applies the Glyph-Aligned ByT5 model for flexible character-level text rendering, and through Scaled ROPE technology achieves generalization to untrained resolutions.

Main Features of Seedream 2.0

• Powerful Bilingual Understanding Capability: Supports high-precision understanding and following of Chinese and English instructions, capable of generating Chinese or English aesthetic expression images with cultural nuances, breaking the dimensional wall between different languages and vision.
• Excellent Text Rendering Capability: Greatly reduces the text corruption rate, font variations appear more natural and aesthetically pleasing, and in the generation of Chinese-style patterns and elements, it can output high-quality results.
• Multi-Resolution Generation Capability: Through the triple-upgraded DiT architecture, it achieves multi-resolution generation and improved training stability, capable of generating image sizes and various resolutions never trained before.
• Reinforcement Learning from Human Feedback (RLHF) Optimization: Through self-developed reward models and feedback learning algorithms, it improves the model's overall performance in image-text alignment, aesthetics, structural correctness, text rendering, and other aspects.

Technical Principles of Seedream 2.0

• Data Preprocessing
  • Data Composition: Carefully curates pre-training data from four parts: high-quality data pairs, distribution maintenance data, knowledge injection data, and targeted supplementary data.
  • Data Cleaning: Through multi-stage filtering methods, ensures data quality and relevance.
  • Active Learning Engine: Optimizes the image classifier to ensure the high quality of the training dataset.
  • Image Annotation: Generates general captions and professional captions, covering multiple description types.
  • Text Rendering Data: Builds a large-scale visual text rendering dataset for text rendering tasks.
• Model Pre-training
  • Diffusion Transformer (DiT): Processes image and text tokens, adopts a scaled version of 2D rotary position embedding (Scaling RoPE), supporting generalization to untrained resolutions.
  • Text Encoder: Self-developed bilingual large language model (LLM), directly learns local knowledge from massive data, supporting high-fidelity image generation.
  • Character-Level Text Encoder: Applies the Glyph-Aligned ByT5 model to achieve flexible character-level text rendering.
• Model Post-training
  • Continual Training (CT): Extends training through high-quality datasets to enhance the aesthetics of generated images.
  • Supervised Fine-Tuning (SFT): Fine-tunes the model with a small amount of high-quality images to enhance artistic aesthetics.
  • Human Feedback Alignment (RLHF): Combines preference data, reward models, and feedback learning algorithms to improve performance in multiple aspects.
  • Prompt Engineering (PE): Rewriting user prompts with a fine-tuned LLM to improve generated image quality.
  • Refiner: Upscales images generated by the base model to higher resolution and fixes structural errors.
• Instruction-based image editing alignment: Seedream 2.0 can adapt to instruction-based image editing models such as SeedEdit, achieving high-quality image editing while preserving high aesthetics and composition fidelity.
• Performance: Seedream 2.0 performs excellently in prompt following, aesthetics, text rendering, and structural correctness. After multiple rounds of RLHF optimization, its outputs are highly aligned with human preferences, with excellent ELO scores.

Seedream 2.0 project address

• Project official website：https://team.doubao.com/zh/tech/seedream
• arXiv technical paper：https://arxiv.org/pdf/2503.07703

How to use Seedream 2.0

• Access the platform to use: VisitDoubao's official website to use or visitJimeng's official website to use.
• Register/Log in: Log in to the Doubao platform with your account.
• Enter prompt: Enter a detailed Chinese or English prompt in the image generation interface to describe the image content you want to generate.
• Select generation mode: Select a suitable generation mode (such as standard generation, HD generation, etc.).
• Adjust parameters: Adjust generation parameters as needed (such as resolution, style, etc.).
• Generate image: Click the generate button and wait for the model to generate the image.
• Download or use the image: The generated image can be downloaded directly or used for further editing.
• Use the API interface
  • Obtain API Key: If you are a developer, you can obtain an API Key through the developer documentation of the Doubao or Jimeng platform.
  • Send request: Use an HTTP request to send the prompt and generation parameters to the Seedream 2.0 API interface.
  • Receive response: The API will return links to the generated images, and you can download or directly use these images.

Application scenarios of Seedream 2.0

• Poster design: Generate attractive posters, support complex text rendering and artistic styles, and generate high-quality poster designs based on user-input prompts.
• Social media content: Generate attractive images for social media platforms, support multiple styles and themes, and help users quickly generate high-quality social media content.
• Video content: Generate cover images, keyframes, etc. for video content, support multiple video styles and scenarios, and generate related images based on video content.
• Painting creation: Generate paintings in various styles, support multiple artistic styles such as oil painting, watercolor, and sketch, and generate high-quality paintings based on user-input prompts.
• Teaching assistance: Generate teaching assistance images, support multiple teaching scenarios, and generate related images based on teaching content.
• Game scene generation: Generate scenes and backgrounds in games, support multiple game styles, and generate related images based on game content.