SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

OmniGen2 - Open-Source Multimodal Generation Model from BAAI

Author: AI小集 · Original source: AI工具集

What is OmniGen2

OmniGen2 is an open-source multimodal generation model launched by the Beijing Academy of Artificial Intelligence (BAAI). It can generate high-quality images based on text prompts and supports instruction-guided image editing, such as modifying backgrounds or character features. OmniGen2 adopts a dual-component architecture, combining a vision-language model (VLM) and a diffusion model to achieve unified processing of multiple generation tasks. Its advantages lie in being open-source and free, high performance, and powerful context generation capabilities, making it suitable for scenarios such as business, creative design, and research and development.

OmniGen2

Main Features of OmniGen2

  • Text-to-Image Generation: Can generate high-fidelity and aesthetically pleasing images based on text prompts. It performs excellently in multiple benchmarks, for example, scoring 0.86 and 83.57 on GenEval and DPG-Bench, respectively.
  • Instruction-Guided Image Editing: Supports complex instruction-driven image modifications, including local modifications (such as changing clothing color) and overall style transfer (such as converting photos into anime style). In image editing tasks, OmniGen2 achieves a balance between editing accuracy and image fidelity in multiple benchmarks.
  • Context Generation: Can process and flexibly combine multiple inputs (such as people, reference objects, and scenes) to generate novel and coherent visual outputs. In the OmniContext benchmark, OmniGen2 surpasses existing open-source models by more than 15% in visual consistency metrics.
  • Visual Understanding: Inherits the powerful image content parsing and analysis capabilities of the Qwen-VL-2.5 foundation model.

Technical Principles of OmniGen2

  • Dual-Path Architecture: OmniGen2 adopts independent text and image decoding paths to process text and image modalities separately. The text generation part is based on the Qwen2.5-VL-3B multimodal large language model (MLLM), and image generation is completed through an independent diffusion Transformer module. This avoids the negative impact of text generation on image quality.
  • Diffusion Transformer: The image generation part adopts a 32-layer diffusion Transformer with a hidden dimension of 2520 and a total parameter count of approximately 4 billion. This module uses the Rectified Flow method for efficient image generation.
  • Omni-RoPE Positional Encoding: OmniGen2 introduces a novel multimodal rotary position embedding (Omni-RoPE), which decomposes positional information into sequence and modality identifiers, two-dimensional height coordinates, and width coordinates. It can precisely encode the information of each position in an image while supporting multi-image spatial localization and identity differentiation.
  • Reflection Mechanism: OmniGen2 designs a dedicated reflection mechanism to improve the quality and consistency of generated images. The model can self-evaluate generated results and make improvements over multiple rounds.
  • Training Strategy: OmniGen2 adopts a staged training method, first pre-training the diffusion model on text-to-image tasks, then introducing mixed-task training, and finally conducting end-to-end training of reflection capabilities.
  • Data Processing: Training data is extracted from videos and undergoes multiple filters, including DINO similarity filtering and VLM consistency checks, to ensure data quality.

Project Address of OmniGen2

  • Project Official Website：https://vectorspacelab.github.io/OmniGen2/
  • Github Repository：https://github.com/VectorSpaceLab/OmniGen2
  • arXiv Technical Paper：https://arxiv.org/pdf/2506.18871

Application Scenarios of OmniGen2

  • Design Concept Generation: Designers can quickly generate design concept images and sketches through simple text descriptions.
  • Story Creation Assistance: Content creators can generate corresponding scene and character images based on the story's plot and character descriptions.
  • Video Production Material Generation: Creators can generate images of various scenes, character actions, and special effects, and then import them into video editing software for use as supplementary materials in making animations, special effects videos, or live-action videos.
  • Game Scene and Character Generation: Developers can quickly generate scenes and characters in games through text descriptions.
  • Teaching Resource Generation: Educators can generate related images and diagrams based on teaching content. For example, when explaining historical events, generate related ancient war scenes or historical figure images.