What Is OmniConsistency
OmniConsistency is an image style transfer model launched by the National University of Singapore that can solve the consistency problem of stylized images in complex scenarios. The model is trained on large-scale paired stylized data, uses a two-stage training strategy to decouple style learning from consistency learning, and maintains the semantic, structural, and detail consistency of images across multiple styles. The model supports seamless integration with LoRA modules of any style, enabling efficient and flexible stylization effects. OmniConsistency demonstrates in experiments performance comparable to GPT-4o while providing greater flexibility and generalization capability.
Main Features of OmniConsistency
- Style Consistency: Maintains the style consistency of images across multiple styles and avoids style degradation.
- Content Consistency: Preserves the semantics and details of the original image during stylization, ensuring the integrity of the content.
- Style Agnosticism: Seamlessly integrates with LoRA (Low-Rank Adaptation) modules of any style, supporting stylization tasks across multiple styles.
- Flexibility: Supports flexible layout control without relying on traditional geometric constraints (such as edge maps, sketches, and pose maps).
Technical Principles of OmniConsistency
- Two-Stage Training Strategy: In the first stage (style learning), multiple style-specific LoRA modules are trained independently, with each module focusing on capturing the unique details of a specific style. In the second stage (consistency learning), a consistency module is trained on paired data, dynamically switching between different style LoRA modules to ensure that the consistency module focuses on structural and semantic consistency and avoids absorbing features of a specific style.
- Consistency LoRA Module: Introduces a Low-Rank Adaptation (LoRA) module into the conditional branch, adjusting only the conditional branch to avoid interfering with the stylization capability of the main network. It uses a causal attention mechanism to ensure that conditional tokens interact internally, while the main branch (noise and text tokens) maintains clean causal modeling.
- Conditional Token Mapping (CTM): Low-resolution conditional images guide high-resolution generation, and a mapping mechanism ensures spatial alignment while reducing memory and computational overhead.
- Feature Reuse: During the diffusion process, intermediate features of conditional tokens are cached to avoid repeated computation and improve inference efficiency.
- Data-Driven Consistency Learning: Builds a high-quality paired dataset containing 2,600 image pairs across 22 different styles, and learns semantic and structural consistency mappings in a data-driven manner.
OmniConsistency Project Address
- GitHub Repository：https://github.com/showlab/OmniConsistency
- HuggingFace Model Library：https://huggingface.co/showlab/OmniConsistency
- arXiv Technical Paper：https://arxiv.org/pdf/2505.18445
- Online Experience Demo：https://huggingface.co/spaces/yiren98/OmniConsistency
Application Scenarios of OmniConsistency
- Artistic Creation: Applies various artistic styles (such as anime, oil painting, sketching, etc.) to images, helping artists quickly generate stylized works.
- Content Generation: In content creation, quickly generates images that match a specific style, improving the diversity and appeal of the content.
- Advertising Design: Generates stylistically consistent images for advertising and marketing materials, improving visual effects and brand consistency.
- Game Development: Quickly generate stylized characters and scenes in games, improving development efficiency.
- Virtual Reality (VR) and Augmented Reality (AR): Generate stylized virtual environments and elements to enhance the user experience.