What is FlexIP
FlexIP is a flexible subject attribute editing framework for image synthesis proposed by Tencent, balancing identity preservation and personalized editing in image generation. The framework adopts a dual-adapter architecture that decouples identity preservation from personalized editing, ensuring identity integrity through high-level semantic concepts and low-level spatial details. A dynamic weight gating mechanism allows users to perform flexible parametric control between identity preservation and style personalization, transforming the traditional binary trade-off into a continuous control surface. FlexIP combines a multimodal data training strategy, optimizing the adapter's identity locking and deformation capabilities based on image and video data respectively, further enhancing generation robustness.
FlexIP's Main Features
- Dual-Adapter Decoupled Design: For the first time, identity preservation (Preservation Adapter) and personalized editing (Personalization Adapter) are explicitly separated. The identity preservation adapter combines high-level semantic concepts with low-level spatial details to ensure identity integrity; the personalization adapter interacts with text and visual CLS tokens, absorbing meaningful visual cues, placing text modifications in a coherent visual context, avoiding feature competition, and achieving more precise control.
- Dynamic Weight Gating Mechanism: Dynamically balances identity preservation and editing strength through continuously adjustable parameters, transforming the traditional binary trade-off into a continuous parametric control surface, supporting flexible control from subtle adjustments to large deformations, allowing users to flexibly adjust the generation effect as needed.
- Modality-Aware Training Strategy: Adaptively adjusts adapter weights based on data characteristics (static images/video frames), strengthens identity locking with image data, optimizes temporal deformation with video data, and enhances generation robustness.
- Cross-Attention Mechanism: The preservation adapter captures multi-granularity visual features (such as facial details) through cross-attention, enhancing identity robustness.
- Dynamic Interpolation: The weight gating mechanism supports users in adjusting adapter contributions in real time, forming a continuous "control surface".
- Multimodal Data Training: Combines image and video data to separately optimize the adapter's identity locking and deformation capabilities.
FlexIP's Performance Comparison
- Quantitative Comparison
- Overall Ranking: On the overall ranking (mRank) metric, FlexIP's score is better than all other methods, indicating that its overall performance across multiple key metrics is the most outstanding.
- Personalization Capability: In the personalization evaluation, FlexIP scores 0.284 on CLIP-T, slightly lower than λ-Eclipse, but λ-Eclipse achieves this at the cost of subject preservation capability. FlexIP can achieve a high level of personalization while maintaining subject features.
- Identity Preservation Capability: In terms of identity preservation, FlexIP achieves high scores of 0.873 and 0.739 on CLIP-I and DINO-I respectively, significantly outperforming other methods, proving its strong advantages in preserving image details and semantic consistency.
- Image Quality: In the image quality evaluation, FlexIP scores 0.598 on CLIP-IQA and 6.039 on aesthetics, indicating that the images it generates are not only high in quality but also have better aesthetics.
- User Study: In user satisfaction evaluations in practical applications, FlexIP performed excellently on both flexibility (Flex) and identity preservation (ID-Pres) metrics, with 60 evaluators unanimously agreeing that images generated by FlexIP best matched the text semantics and best preserved the subject's features.
- Qualitative Comparison
- Fidelity: FlexIP-generated images perform excellently in terms of fidelity, highly restoring the subject's features and details from the reference image. Even during personalized editing, they maintain the high quality and realism of the image.
- Editability: FlexIP has a significant advantage in editability, generating diverse editing results according to different text instructions and meeting users' personalized needs in various scenarios.
- Identity Consistency: In terms of identity consistency, FlexIP can stably maintain the subject's features across different reference images, and even during substantial deformation or stylized editing, it ensures the subject's identity consistency, avoiding the identity mutation issues commonly seen in traditional methods.
- Comparison with Existing Methods: In qualitative comparisons with five state-of-the-art methods, FlexIP-generated images show significant improvements in fidelity, editability, and identity consistency, better meeting users' needs for high-fidelity personalized image generation.
FlexIP Project Address
- Project Website：http://flexip-tech.github.io/flexip/#/
- arXiv Technical Paper：https://arxiv.org/pdf/2504.07405
FlexIP Application Scenarios
- Artistic Creation: FlexIP can flexibly perform personalized editing of images according to artists' needs while maintaining the subject's identity features.
- Advertising Design: In the field of advertising design, FlexIP can help designers quickly generate image content that meets brand needs. Through the dynamic weight gating mechanism, designers can flexibly adjust the style, scene, and details of advertising images while maintaining brand image.
- Film and Television Production: FlexIP can be used for visual effects and character design in film and television production. It can flexibly adjust the character's appearance while maintaining the character's identity consistency.
- Game Development: In game development, FlexIP can be used for generating and editing characters and scenes. Developers can use this framework to quickly generate diverse character images while maintaining the character's core features.