SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

Live Avatar - A Real-Time Digital Human Model Open-Sourced by Alibaba and Universities

Author: AI小集 · Original source: AI工具集

What is Live Avatar

Live Avatar is a real-timedigital humanmodel launched by Alibaba in collaboration with multiple universities. It can generate digital human videos of unlimited duration driven by audio and video. Live Avatar relies on a 14-billion-parameter diffusion model to achieve real-time streaming generation at 20 FPS, supports stable output for over 10,000 seconds, and avoids facial drift and color distortion during long-duration generation. It can be used in various scenarios such as e-commerce livestreaming, news broadcasting, and virtual assistants, providing high-fidelity, low-latency interactive experiences.

Live Avatar

Main Features of Live Avatar

  • Real-Time Audio-Video Driving: Captures the user's voice and facial expressions through a microphone and camera, driving the digital human to respond synchronously in lip movements and expressions, enabling low-latency face-to-face interaction.
  • Unlimited-Duration Stable Generation: Supports continuous video generation for over 10,000 seconds, during which the digital human's facial features, skin tone, style, and other characteristics remain consistent, avoiding identity drift and quality degradation during long-duration generation.
  • High-Fidelity Image Quality: Based on a 14-billion-parameter diffusion model, it generates detailed, clear, and natural digital human visuals.
  • Streaming Generation: Uses streaming processing technology to support continuous extension of real-time video streams, suitable for real-time application scenarios such as livestreaming.

Technical Principles of Live Avatar

  • Diffusion Model Optimization：Using a 14-billion-parameter diffusion model, through Distribution Matching Distillation technology, it transforms a multi-step bidirectional diffusion model into an efficient 4-step streaming diffusion model, greatly improving generation speed.Adopts Timestep-forcing Pipeline Parallelism technology to decouple the denoising stage and distribute it across multiple devices, achieving linear acceleration and supporting large-scale parallel computing.
  • Unlimited-Duration Generation Mechanism：
    • Rolling RoPE (Relative Positional Encoding): Dynamically updates the relative positional encoding of reference frames, maintaining the relative positional relationship between generated frames and reference frames, avoiding identity drift.
    • Adaptive Attention Sink: Replaces the initial reference frame with generated frames, eliminating persistent factors that cause distribution drift and maintaining distribution consistency of generated frames.
    • Historical Interference Mechanism: Injects noise into the KV cache to simulate errors during inference, guiding the model to extract motion information from historical frames and stable details from reference frames, avoiding error accumulation.
  • Real-Time Interaction Technology：Combines microphone and camera input to drive the digital human model in real time through audio and video signals, achieving low-latency interactive responses.Optimizes generation speed and latency, achieving a real-time generation speed of 20 FPS, with first-frame latency of only 2.89 seconds, ensuring a smooth interactive experience.

Live Avatar Project Address

  • Project Website：https://liveavatar.github.io/
  • GitHub Repository：https://github.com/Alibaba-Quark/LiveAvatar
  • HuggingFace Model Library：https://huggingface.co/Quark-Vision/Live-Avatar

Application Scenarios of Live Avatar

  • E-Commerce Livestreaming: Live Avatar can achieve 24/7 uninterrupted product display and explanation, reducing labor costs and improving livestreaming efficiency.
  • News Broadcasting: The model supports automatically generating virtual anchor news broadcasts, quickly generating multilingual videos, and improving news timeliness.
  • Entertainment Livestreaming: Virtual idols use Live Avatar for real-time interactive performances, host online concerts and other activities, and enhance audience participation.
  • Intelligent Customer Service: Digital human customer service can answer user questions in real time, provide 24-hour uninterrupted service, and improve the user experience.
  • Bank Teller: Virtual financial advisors provide standardized business consultation, assist human services, and improve bank operational efficiency.