What is PartCrafter
PartCrafter is an advanced3D generation model, capable of generating multiple semantically distinct and geometrically diverse 3D meshes from a single RGB image. It represents each 3D part through compositional latent space and uses a hierarchical attention mechanism to transmit information within and between parts, ensuring the generated 3D model has global consistency. The model is based on a pretrained 3D mesh diffusion transformer (DiT), inheriting its weights, encoder, and decoder, further enhancing generation capability. It enables end-to-end part-aware generation, supporting generation from a single object to complex multi-object scenes. Experiments show that the model surpasses existing methods in generating decomposable 3D meshes, and can accurately generate parts that are not directly observable in the input image.
Main features of PartCrafter
- Joint generation of multiple parts: PartCrafter can generate multiple parts and objects at once from a single image, each part having clear semantics and unique geometric form.
- Part-aware generation: The model represents each 3D part through compositional latent space and adopts a hierarchical attention mechanism to transmit information within and between parts.
- End-to-end generation: PartCrafter supports end-to-end generation from a single image to complex multi-object scenes. Users only need to provide one image, and the model can generate a complete 3D scene, including multiple independent parts and objects.
- Part-level editing: The generated 3D model supports part-level editing and modification. Users can perform individual operations on generated parts, such as adjusting position, rotation, scaling, etc., enabling more flexible 3D design and creation.
- Complex scene modeling: PartCrafter is suitable for modeling complex scenes, such as mechanical structures, architectural models, etc. It can generate multiple parts and ensure the collaborative relationships between these parts, generating more realistic and complex 3D scenes.
- Pretrained model support: The model is based on a pretrained 3D mesh diffusion transformer (DiT), inheriting its weights, encoder, and decoder. This pretraining mechanism further enhances the model's generation capability and efficiency.
Technical principles of PartCrafter
-
Compositional latent space: Each 3D part is represented by a set of decoupled latent tokens, allowing parts to evolve independently during generation and preserving part-level details.
-
Hierarchical attention mechanism: Supports structured information flow within a single part and between all parts, ensuring global consistency during generation.
-
Based on pretrained 3D mesh diffusion transformer (DiT): PartCrafter inherits pretrained weights, encoder, and decoder, further enhancing the model's generation capability.
-
End-to-end part-aware generation: Under the condition of a single image, PartCrafter can simultaneously denoise multiple 3D parts, achieving end-to-end part-aware generation from individual objects to complex multi-object scenes.
Project address of PartCrafter
- Project official website：https://wgsxm.github.io/projects/partcrafter/
- Github repository：https://github.com/wgsxm/PartCrafter
- arXiv technical paper：https://arxiv.org/pdf/2506.05573
Application scenarios of PartCrafter
- Game development: PartCrafter can quickly generate high-quality 3D game assets, including characters, props, and scenes. Through a single image or text description, developers can obtain a large number of 3D models in a short time, accelerating the game development process.
- Architecture and Interior Design: In the fields of architecture and interior design, PartCrafter can be used to quickly generate architectural models and interior decoration schemes. Designers can quickly generate 3D models by uploading architectural drawings or design sketches for visual presentation of schemes, helping clients better understand the design intent.
- Film and Television Production: PartCrafter's ultra-high-definition geometric detail modeling capability and multi-view input PBR model generation function can provide high-quality 3D model materials for film and television production teams.
- Education: In the field of education, PartCrafter can be used to display complex scientific concepts, such as molecular structures and human anatomy, enhancing students' understanding and learning outcomes.
- Augmented Reality/Virtual Reality: The realistic 3D models generated by PartCrafter help build more realistic AR/VR experiences. 3D models generated from a single image can be directly used to construct virtual scenes, bringing users a more immersive experience.