What is HoloPart
HoloPart is a novel diffusion model launched by the University of Hong Kong and the VAST team, supporting the decomposition of 3D objects into complete, editable semantic parts, even when the parts are occluded. HoloPart is based on a two-stage method, using local attention and global context attention mechanisms to ensure consistency between part details and the overall shape. HoloPart significantly outperforms existing methods on the ABO and PartObjaverse-Tiny datasets, providing new possibilities for downstream applications such as geometry editing, material editing, and animation production.
HoloPart's main features
- 3D part implicit segmentation: Identifies visible surface fragments, supports completing occluded parts, and generates complete 3D parts.
- Geometric super-resolution: Supports super-resolution reconstruction of geometric details.
- Downstream application support: Supports a variety of downstream applications, including geometry editing, material editing, animation production, and geometry processing.
HoloPart's technical principles
- Two-stage method：
- Initial segmentation: Uses existing 3D part segmentation techniques (such as SAMPart3D) to obtain initial, incomplete part fragments (surface fragments).
- Part completion: Based on PartComp (a diffusion model-based network), completes the fragments into complete 3D parts.
- Diffusion model: PartComp is a diffusion model-based network that captures fine-grained geometric details of parts to ensure that the local features of the parts are accurately restored. It uses contextual information of the overall shape to ensure that the completed parts remain geometrically and semantically consistent with the overall shape.
- Data pretraining and fine-tuning: Uses a variational autoencoder (VAE) and a diffusion model to pretrain on large-scale complete 3D shape data to learn general 3D shape representations. The pretrained model is fine-tuned on limited part data to adapt to the part completion task, overcoming the challenge of data scarcity.
HoloPart's project address
- Project official website：https://vast-ai-research.github.io/HoloPart/
- GitHub repository：https://github.com/VAST-AI-Research/HoloPart
- HuggingFace model library：https://huggingface.co/VAST-AI/HoloPart
- arXiv technical paper：https://arxiv.org/pdf/2504.07943
- Online experience Demo：https://huggingface.co/spaces/VAST-AI/HoloPart
HoloPart's application scenarios
- Geometry editing: Modify the size, shape, and position of parts to meet design needs.
- Material assignment: Add different materials to parts to enhance visual effects.
- Animation production: Allow parts to move independently, such as wheels rotating, improving animation flexibility.
- Geometry processing: Optimize the mesh division of parts to improve model quality.
- Data generation: Provide high-quality part data for 3D model training, enriching creative materials.