What is PrimitiveAnything
PrimitiveAnything is a new 3D shape generation framework jointly developed by Tencent's Artificial Intelligence Platform Department (AIPD) and Tsinghua University. It decomposes complex 3D shapes into simple primitive elements, generates these primitives in an autoregressive manner, and finally reassembles them into complete 3D shapes. The core advantages of the framework lie in its high-quality generation results, strong generalization ability, and high efficiency.
Main Features of PrimitiveAnything
- High-Quality 3D Primitive Assembly Generation: It can generate high-quality 3D primitive assemblies that are geometrically faithful to the original model and also align with humans' intuitive understanding of shapes.
- Diverse 3D Content Creation: It supports generating 3D content from text or image conditions, providing users with a flexible way to create.
- Efficient Storage and Editing: Because primitive representations are used, the generated 3D models are more efficient in storage and are also easy to edit and adjust.
- Autoregressive Transformer Architecture: It generates 3D primitives frame by frame through an autoregressive Transformer, can handle primitive sequences of different lengths, and can be easily extended to new primitive types.
- Unambiguous Parameterization Scheme: By eliminating ambiguity in parameterization, it ensures that the model maintains stability and accuracy during training and generation.
- Geometric Fidelity and Semantic Consistency: During the generation process, PrimitiveAnything maintains high geometric fidelity and can generate semantic decomposition results that align with human cognition.
- Modular Design: The framework's modular design supports seamless integration of new primitive types without modifying the architecture, and can adapt to different primitive representations.
Technical Principles of PrimitiveAnything
- Unambiguous Parameterization Scheme
- Unified Representation: Multiple types of primitives (such as cubes, elliptical cylinders, and ellipsoids) are used to jointly represent 3D shapes under a unified parameterization scheme, and attributes such as the type, position, rotation, and scale of each primitive are encoded and input into the model.
- Eliminating Ambiguity: To address the inherent ambiguity in parameterization (for example, different parameter combinations may produce the same shape), the research team developed a comprehensive set of rules. By analyzing the symmetry of primitives, it selects the set with the smallest L1 norm of rotation parameters as the unique representation, ensuring the stability and accuracy of the training process.
- Autoregressive Transformer Architecture
- Shape Conditioning: The framework adopts a decoder-based Transformer architecture and can generate primitive sequences of variable length according to shape features. First, a point cloud encoder extracts the feature representation of the 3D shape, and then the autoregressive Transformer model receives these shape features and the previously generated primitive information to predict the features of the next primitive.
- Cascaded Decoder: To model the dependencies among primitive attributes, the framework uses a cascaded decoder to sequentially predict attributes such as the type, position, rotation, and scale of primitives. It captures the natural correlations among primitive attributes; for example, the primitive type affects its possible position, rotation, and scale parameters, which is consistent with human assembly logic.
- Autoregressive Generation Process
- Sequence Generation: Reconceive the entire primitive abstraction process as a sequence generation task, where the model takes a point cloud as input conditioning and then autoregressively generates a sequence of primitives until an end token is predicted.
- Training objective: During training, cross-entropy loss, Chamfer distance (for reconstruction accuracy), and Gumbel-Softmax (for differentiable sampling) are combined until the end-of-sequence token is generated. It can decompose complex 3D shapes flexibly and in a human-like manner.
PrimitiveAnything project address
- Project website：https://primitiveanything.github.io/
- Github repository：https://github.com/PrimitiveAnything/PrimitiveAnything
- HuggingFace model library：https://huggingface.co/hyz317/PrimitiveAnything
- arXiv technical paper：https://arxiv.org/pdf/2505.04622
PrimitiveAnything application scenarios
- 3D modeling and design: PrimitiveAnything can quickly generate the "geometric skeleton" of complex 3D models, and designers can focus on detailed refinement on this basis, improving work efficiency and design speed.
- Game asset generation: Game designers can use PrimitiveAnything to quickly generate various game scenes and character models, improving development efficiency. Players can also create new characters or props by simply assembling geometric blocks, which the AI automatically optimizes and adjusts and seamlessly integrates into the physics engine.
- User-generated content (UGC): The framework supports generating 3D content from text or image input, and users can easily edit the generated results, providing new possibilities for UGC in games.
- Virtual reality (VR) and augmented reality (AR): In virtual reality and augmented reality environments, PrimitiveAnything can quickly generate realistic 3D objects, enhancing the user experience.