What is LHM
LHM (Large Animatable Human Reconstruction Model) is an animatable 3D human model reconstructed from a single image, launched by Alibaba Tongyi Lab. Based on a multimodal Transformer architecture, it fuses 3D geometric features and 2D image features, uses attention mechanisms to preserve clothing geometry and texture details, and introduces a head feature pyramid encoding scheme to enhance facial detail recovery. LHM represents the reconstructed 3D model in the form of 3D Gaussian Splatting, supporting real-time rendering and pose-controlled animation. The model can generate high-quality animatable 3D human models within seconds, suitable for immersive applications such as AR/VR.
Main Features of LHM
- Fast Reconstruction: Completes the conversion from a single image to a 3D animatable model within seconds, without complex post-processing.
- High-Fidelity Details: Accurately preserves key information such as clothing textures and facial details, generating high-quality 3D models.
- Real-Time Animation: Supports real-time animation rendering based on pose control, suitable for immersive applications (such as AR/VR).
- Strong Generalization Ability: Performs well on in-the-wild images, adapting to various scenes and poses.
Technical Principles of LHM
- Multimodal Transformer Architecture: Based on the Transformer architecture, it fuses 3D geometric features (surface points sampled from the SMPL-X template) and 2D image features (extracted from a pre-trained vision Transformer), effectively processing geometric and visual information. A multi-scale feature extraction scheme designed for the head region aggregates features at different levels to enhance facial detail recovery.
- 3D Gaussian Splatting Representation: Represents the 3D model based on 3D Gaussian Splatting, supporting real-time, high-quality rendering. The network directly predicts the parameters of the Gaussian splats (such as position, rotation, scale, color, etc.), enabling fast conversion from the input image to the 3D model.
- Self-Supervised Learning: Trained on large-scale video data, the model is optimized using rendering losses and regularization terms, without relying on scarce 3D scan data. During training, regularization terms of 'as close as possible' and 'as spherical as possible' are introduced to maintain the geometric plausibility of the 3D model.
- Real-Time Animation Support: Based on SMPL-X skeleton parameters, the reconstructed 3D model is deformed to the target pose, supporting real-time pose-controlled animation. The entire reconstruction and animation process is completed in a single forward pass, suitable for real-time applications.
LHM Project Address
- Project Website：https://lingtengqiu.github.io/LHM/
- GitHub Repository：https://github.com/aigc3d/LHM
- arXiv Technical Paper：https://arxiv.org/pdf/2503.10625
- Online Experience Demo：https://huggingface.co/spaces/DyrusQZ/LHM
Application Scenarios of LHM
- Virtual Reality (VR) and Augmented Reality (AR): Quickly transforms photos into animatable 3D virtual characters, enhancing immersion and interactivity.
- Game Development: Quickly generates high-quality 3D character models, supports real-time animation, and improves development efficiency and gaming experience.
- Film and Television Production: Used in special effects production and animated films to quickly generate character models, improving production efficiency and quality.
- Social Media and Content Creation: Users can generate 3D avatars for social media, and creators can quickly generate 3D characters for short videos, etc.
- Education and Training: Create virtual teachers or teaching assistants for online education, and generate 3D models for simulation training in fields such as healthcare and the military.