AnimaX - A 3D animation generation framework jointly launched by Beihang University, Tsinghua University, and others

What is AnimaX

AnimaX is an efficient 3D animation generation framework jointly launched by Beihang University, Tsinghua University, the University of Hong Kong, and others. It combines the motion priors of video diffusion models with skeleton-based animation structures. The framework can effectively transfer motion knowledge from videos to the 3D domain and supports diverse articulated meshes with arbitrary skeleton structures. AnimaX represents 3D motion as multi-view, multi-frame 2D pose maps and uses a joint video-pose diffusion model, combined with shared positional encoding and modality-aware embeddings, to ensure spatiotemporal alignment between videos and pose sequences, achieving high-quality 3D animation generation. The framework's training dataset contains 160,000 rigged sequences. AnimaX achieves industry-leading performance in generalization, motion fidelity, and efficiency, providing a scalable solution for category-agnostic 3D animation.

Main Features of AnimaX

• Arbitrary Skeleton Structure Support: AnimaX supports processing 3D models with different skeletal topologies, applicable to various characters and objects, such as humans, animals, furniture, etc.
• Text-Driven Animation: Users specify animation content based on simple text descriptions, and AnimaX generates corresponding animation sequences according to text prompts.
• Multi-View Consistency: The generated animations remain consistent across multiple viewpoints, ensuring coherence and realism when observed from different angles.
• Efficient Generation: AnimaX is based on a feedforward method and generates high-quality 3D animations in a short time, greatly improving animation production efficiency.

Technical Principles of AnimaX

• Joint Video-Pose Diffusion Model: AnimaX represents 3D motion as multi-view, multi-frame 2D pose maps. The model leverages the powerful motion priors of video diffusion models while maintaining compatibility with 3D skeletal animation. With the joint video-pose diffusion model, AnimaX can simultaneously generate videos and corresponding 2D pose sequences. The joint generation strategy ensures spatiotemporal alignment between videos and poses.
• Shared Positional Encoding and Modality-Aware Embeddings: To ensure spatiotemporal alignment between videos and pose sequences, AnimaX introduces shared positional encoding. This encoding mechanism enables the model to better understand and align spatial information across different modalities (RGB videos and pose maps). Modality-aware embeddings are used to distinguish data from different modalities, ensuring the model correctly handles the differences between RGB videos and pose maps.
• Multi-View Consistency: AnimaX introduces a multi-view attention mechanism and camera-conditioned embeddings to ensure that the generated videos remain consistent across multiple viewpoints. This allows the model to learn spatial correspondences between different viewpoints and generate consistent multi-view videos.
• 3D Motion Reconstruction and Animation: The generated multi-view pose sequences are converted into 3D joint positions based on triangulation. Inverse kinematics is used to convert 3D joint positions into joint angles to drive the animation of 3D models.
• Large-Scale Dataset Training: AnimaX is trained on a newly curated dataset containing 160,000 rigged sequences. The data covers multiple categories, such as humans, animals, furniture, etc., ensuring the model has good generalization ability.

Project Address of AnimaX

• Project Website：https://anima-x.github.io/
• GitHub Repository：https://github.com/anima-x/anima-x
• arXiv Technical Paper：https://arxiv.org/pdf/2506.19851

Application Scenarios of AnimaX

• Game Development: Quickly generate animations for characters and environments, improving development efficiency and content richness.
• Film and Television Production: Used in animated films, visual effects, and animation generation for virtual characters to enhance visual effects.
• Virtual Reality (VR) and Augmented Reality (AR): Generate animations of virtual characters and dynamic environments to improve immersion and interactive experience.
• Advertising and Marketing: Create dynamic advertisements and product demonstration animations to attract audience attention and increase engagement.
• Education and Training: Generate animations for virtual experiments and simulation training to improve teaching and training effectiveness.