What is BAG
BAG (Body-Aligned 3D Wearable Asset Generation) is an innovative 3D wearable asset generation technology jointly proposed by The Chinese University of Hong Kong and Tencent. By combining multi-view image diffusion models and ControlNet, and utilizing human body shape and pose information, it automatically generates 3D wearable assets that perfectly fit the human body, such as clothing and accessories. The core of BAG lies in a multi-view generator and a 3D diffusion model. First, multi-view images aligned with the human body are generated through a multi-view image diffusion model, and then these images are converted into 3D assets based on a 3D diffusion model. Through physical simulation and optimization, the generated assets can naturally fit the human body and avoid penetration issues.
Main Features of BAG
- Multi-View Image Generation: Through a multi-view image diffusion model, multi-view images aligned with the human body are generated based on the body's shape and pose information.
- 3D Asset Generation: The multi-view images are input into a 3D diffusion model to generate high-quality 3D wearable assets.
- Diversity and Personalization: Supports generation from a single asset to combinations of multiple components, meeting diverse needs in different scenarios.
- Efficient Adaptation: Can quickly generate assets that adapt to different human body models, suitable for fields such as virtual clothing try-on, game development, VR, and AR.
Technical Principles of BAG
- View Image Diffusion Model: BAG first builds a diffusion model from a single image to consistent multi-view images, trained on the large-scale Objaverse dataset to achieve diversity and generalization ability. The model uses ControlNet to guide the generation of multi-view images aligned with the human body. The control signal utilizes the multi-view 2D projections of the target human body, where pixel values represent the XYZ coordinates of the human body surface in normalized space.
- 3D Shape Generation: The generated multi-view images are input into a local 3D diffusion model to generate the 3D shape of the asset. Through multi-view silhouette supervision, the similarity transformation (Sim3) is recovered to ensure that the generated 3D asset is spatially aligned with the human body model.
- Physical Simulation and Optimization: To ensure that the generated 3D assets can naturally fit the human body, BAG uses physical simulation technology to solve the penetration problem between the asset and the body. A proxy mesh preserves the basic geometry of the asset as a representation for cloth simulation, ultimately achieving penetration-free alignment.
BAG Project Address
- Project Official Website：https://bag-3d.github.io/
- arXiv Technical Paper：https://arxiv.org/pdf/2501.16177
Application Scenarios of BAG
- Virtual Clothing Try-On: BAG can quickly generate matching 3D clothing and accessories based on the user's body model. Users can try on clothing of different styles and designs in real time in a virtual environment, improving the shopping experience and reducing return rates caused by unsuitable sizes.
- Game Development: In game development, BAG can be used to generate diverse clothing and accessories for game characters, support personalized customization, and enrich players' character experience.
- Virtual Reality (VR) and Augmented Reality (AR): The 3D assets generated by BAG can be seamlessly integrated into VR and AR applications, providing users with an immersive clothing try-on and design experience, such as trying on clothing of different styles in a virtual showroom.
- Fashion Design: Designers can use BAG to quickly generate and iterate 3D clothing models, accelerating the design process, previewing design effects in a virtual environment, and reducing design costs.