What is Wear-Any-Way
Wear-Any-Way is an innovativevirtual try-onframework launched by Alibaba's Pailitao team. Through a sparse correspondence alignment mechanism, it achieves high-fidelity and customizable virtual try-on results. Users can generate realistic try-on images and, through simple click-and-drag operations, precisely control how clothing is worn, such as rolling up sleeves or adjusting how a coat opens and closes, bringing a completely new interactive experience to virtual try-on. Wear-Any-Way supports multiple input types, including single-item try-on and multi-item coordinated try-on, and can be applied to various scenarios such as model-to-model and model-to-street.
Wear-Any-Way's main features
- Freely customizable try-on: Users can freely customize the clothing on a virtual model according to their own preferences and needs, including adjusting the size, color, style, and detail elements of the clothing, such as sleeve length, collar type, waistline, and more. They can try different styling combinations, such as inner layers, outerwear, and accessories, to create a unique outfit effect.
- High-fidelity rendering: Based on advanced image rendering technology, it ensures that the virtual try-on effect is close to the real wearing experience. Through high-precision model construction and light-and-shadow processing, the system can present the material texture, texture details, and morphological changes of clothing after wearing, allowing users to experience more realistic and vivid visual effects during the try-on process.
- Multi-item clothing coordinated try-on: Supports virtual try-on of a single item of clothing, and supports providing both tops and bottoms at the same time to achieve a complete outfit try-on effect.
- Multi-scenario try-on: Can support various subtasks. Model-to-model try-on, coordinated try-on of multiple items of clothing, or complex scenarios such as street shots and sitting poses.
- Intelligent recommendation and coordination: Based on Alibaba's big data analytics and AI technology, Wear-Any-Way can intelligently recommend suitable clothing styles and coordination schemes according to users' shopping history, browsing records, and personal preferences.
Wear-Any-Way's technical principles
- Sparse correspondence alignment mechanism: By learning a series of replaceable point embeddings and injecting them into the model, it achieves precise control over clothing details. Users can customize the on-body effect of clothing through simple click-and-drag operations, such as dragging sleeves to roll them up, dragging a coat to open it, or controlling the style of folds by clicking.
- Dual U-Net structure: The entire framework consists of two U-Nets. The Reference U-Net takes clothing images as input and extracts fine-grained features; the Main U-Net is responsible for generating try-on results, requiring the person image (masked), clothing mask, and latent noise as input, and is controlled by pose through an additional pose encoder.
- Point-based control: Implemented by a point embedding network and a sparse correspondence alignment module. The flame and snowflake symbols represent trainable parameters and frozen parameters, respectively.
Wear-Any-Way's project address
- Project official website：https://mengtingchen.github.io/wear-any-way
- arXiv technical paper：https://arxiv.org/pdf/2403.12965
Wear-Any-Way's application scenarios
- Virtual fitting room: Consumers can see the effect of clothes worn on a model in a virtual fitting room, and can adjust the clothes to their preferred style through interactive operations.
- Fashion design and display: Designers and brands can more flexibly display the effect of different models wearing the same clothing, and better evaluate the adaptability and diversity of the clothing.
- Personalized customization: Users can adjust how clothing is worn according to their own preferences and needs, such as rolling up sleeves, opening a jacket, adjusting folds, etc.
- E-commerce and retail: Merchants can more intuitively display the effect of clothing on models, and users can see the effect of clothing in different scenarios in a virtual environment, improving the consumer shopping experience.