Pixal3D - A single-image 3D generation project open-sourced by Tencent in collaboration with Tsinghua University and others

What is Pixal3D

Pixal3D is a single-image 3D generation project launched by Tencent ARC Lab in collaboration with Tsinghua University and Victoria University of Wellington. Pixal3D explicitly lifts pixel features into three-dimensional space through back-projection, establishing direct pixel-to-3D correspondences, and can generate 3D assets with near-reconstruction-level fidelity, featuring fine geometric structures and PBR textures, suitable for scenarios such as games, film and television, and digital content creation.

Pixal3D's main features

• Single-image to high-fidelity 3D model: A single RGB image can generate a 3D asset with fine geometry and PBR textures.
• Pixel-level 3D correspondence reconstruction: Establishes an explicit mapping between original pixels and three-dimensional space through back-projection, preserving the local details of the input image.
• PBR material generation: Simultaneously outputs texture maps such as base color, roughness, and metallic that conform to physically based rendering pipelines.
• Reconstruction-level fidelity output: The generated results approach multi-view reconstruction standards in geometric accuracy and texture consistency.

Pixal3D's technical principles

• Back-projected feature lifting: Pixal3D's core innovation explicitly lifts the pixel features of 2D images into 3D space through a back-projection mechanism. Unlike traditional methods that rely on implicit neural representations or pure optimization, it establishes a direct three-dimensional correspondence for each pixel, precisely mapping the color, edge, and semantic information in the image onto the geometric surface.
• Explicit pixel-3D correspondence: The project builds an explicit correspondence framework that enables 2D pixel coordinates to be directly associated with 3D vertices or surface points. Explicit correspondence avoids information loss during the intermediate feature distillation process and ensures that the contours and texture details of the generated model are highly consistent with the input image.
• Joint modeling of PBR textures and fine geometry: While reconstructing geometry, Pixal3D adopts a joint optimization strategy to generate physically based rendering (PBR) materials. By binding material attributes (such as albedo and roughness) to the geometric surface and using pixel-level correspondences for texture mapping, it achieves high-quality asset output that can be imported into mainstream engines.

How to use Pixal3D

• Visit the project homepage to experience the interactive Demo: Visit the Pixal3D Demo https://huggingface.co/spaces/TencentARC/Pixal3D，进入在线体验界面，上传单张图像可加载并交互式预览生成的 3D model.
• Download the pretrained model: Obtain the officially released pretrained weight files.
• Clone the code repository and configure the environment: Enter the GitHub repository, clone the project locally, and install the dependency environment according to the README to ensure support for the deep learning frameworks required for sparse SDF feature volume computation and pixel back-projection.
• Prepare the input image and run inference: Select an RGB image with a clear foreground and a well-defined target subject as input and run the inference script; Pixal3D's image back-projection conditioner will explicitly lift multi-scale 2D features into a 3D feature volume, generating pixel-aligned coarse structure and high-fidelity detail latent representations.
• Decode and output the 3D asset: Decode the predicted sparse latent representation into a high-fidelity mesh model through a VAE, simultaneously obtaining fine geometric structures and PBR texture maps, with support for direct export to industry-standard formats.
• Import into a creation engine for use: Import the generated textured 3D model into mainstream DCC tools and game engines such as Blender, Unity, or Unreal Engine.

Pixal3D's core advantages

• Reconstruction-level fidelity: Generation quality approaches multi-view reconstruction results, with geometric and texture details significantly outperforming single-image generation baselines.
• Explicit pixel correspondence: The back-projection mechanism establishes a direct pixel-to-3D mapping, reducing information loss caused by feature compression.
• Native PBR material support: Outputs physically based rendering textures that meet industry standards in one pass, with no need for manual post-production material maps.
• Recognized by top academic conferences: The paper was accepted to SIGGRAPH 2026, and the technical approach has been validated through rigorous peer review.

Pixal3D project address

• Project website：https://ldyang694.github.io/projects/pixal3d/
• GitHub repository：https://github.com/TencentARC/Pixal3D
• HuggingFace model library：https://huggingface.co/TencentARC/Pixal3D
• arXiv technical paper：https://arxiv.org/pdf/2605.10922

Comparison of Pixal3D with similar competitors

Pixal3D application scenarios

• Rapid game asset production: Directly generate textured 3D models that can be imported into Unity / Unreal from concept art or reference photos.
• Film and digital content production: Quickly provide high-fidelity 3D props and base character models for previsualization and scene layout.
• E-commerce 3D display: Automatically convert product flat images into interactive rotatable 3D display models, enhancing the online shopping experience.
• Architecture and industrial design preview: Generate 3D prototypes based on a single product photo, accelerating design review and iteration processes.