What is PSHuman
PSHuman is an advanced single-image 3D portrait reconstruction technology. Based on a cross-scale multi-view diffusion model, it requires only one photo to generate highly realistic 3D portrait models, including fine facial expressions and full-body poses. Its core advantage lies in its ability to simultaneously model the joint probability distribution of global shape and local details, avoiding geometric distortion while also maintaining consistency of body shape across different views. PSHuman efficiently recovers realistic textured human meshes through explicit human sculpting technology. It performs excellently on multiple datasets, with outstanding geometric details, texture fidelity, and generalization ability.
PSHuman's main features
-
Single-image 3D portrait reconstruction: Users only need to provide one portrait photo, and PSHuman can quickly generate a detailed 3D model, greatly improving the efficiency of 3D human body modeling.
-
Multi-view diffusion technology: Based on an advanced multi-view diffusion model, it generates human body images from multiple angles from a single-view photo, ensuring that each angle reflects the original subject as realistically as possible, solving the challenge of single-view image reconstruction.
-
High-fidelity facial details: Through the cross-scale diffusion method, it simultaneously takes into account both overall body shape and local facial feature details, avoiding geometric deformation. The final generated 3D model performs excellently in facial features, with a high degree of realism and fineness.
-
Combined with the SMPL-X human body model: It combines the SMPL-X parametric human body model, can represent human bodies with different poses and body types, generates more natural and realistic human movements and forms, and further improves the realism of the 3D model.
-
Background removal: Supports the use of Clipdrop or rembg tools to remove the background from portrait photos, simplifying the subsequent processing workflow.
-
Structured output: The generated 3D models and rendered videos are saved as structured files, making it easy for users to view and share.
-
SMPL-free version: Multi-view generation can be performed without SMPL conditions, making it suitable for portraits in general poses and expanding its application scenarios.
PSHuman's technical principles
-
Body + face enhancement and multi-view generation: The system first feeds the input photo and a predicted human skeleton model (SMPL-X) into a multi-view image diffusion model. The model can generate full-body images from 6 different angles as well as high-precision local facial images, ensuring that both full-body and facial details are as realistic as possible.
-
Cross-scale diffusion method: It can simultaneously model the joint probability distribution of global full-body shape and local facial features, avoiding geometric deformation, so that the final generated 3D model performs excellently in facial features, with a high degree of realism and fineness.
-
Explicit human sculpting based on SMPL-X: Based on the generated normal maps (showing 3D shape details) and color maps (texture details), it uses differentiable rendering technology to adjust, deform, and re-refine the SMPL-X model. This step combines multi-view generation and 3D sculpting technology, enabling PSHuman to efficiently "sculpt" a 3D human model with realistic textures from a single photo.
-
Multi-view image fusion: It uses differentiable rendering technology to fuse multi-view color images, reducing generation inconsistency, and projects the fused texture onto the reconstructed 3D mesh to achieve texture mapping.
-
Optimization and reconstruction module: Geometry optimization is performed by optimizing SMPL-X parameters to align multi-view images. The mesh is sculpted using differentiable rasterization techniques to match the observed normal maps.
PSHuman's project address
- Project website：https://penghtyx.github.io/PSHuman
- GitHub repository：https://github.com/pengHTYX/PSHuman
- arXiv technical paper：https://arxiv.org/pdf/2409.10141
PSHuman's application scenarios
-
Film and television production and game development: In film and television special effects and game scenes, PSHuman can quickly generate realistic 3D character models, improve production efficiency, reduce production costs, and add more realism and fine detail to film, television, and games.
-
Virtual reality and augmented reality: In the fields of VR and AR, through the 3D human models generated by PSHuman, users can interact with virtual characters in virtual environments and gain an immersive experience. In addition, it can also be applied to virtual fitting, virtual makeup, and more, providing users with convenient and personalized services.
-
Fashion design: The fashion industry can use PSHuman technology to quickly generate clothing try-on effects, carry out clothing design and display, accelerate the design process, and improve the user experience.
-
Personalized customization: Generate personalized 3D models from a single photo, which can be used to customize personalized products, such as custom figurines or game characters, meeting users' needs for personalized and customized content.