What is LLIA
LLIA (Low-Latency Interactive Avatars) is a real-time audio-driven portraitvideo generationframework launched by Meituan based on diffusion models. The framework drives the generation of virtual avatars based on audio input, supporting low-latency, high-fidelity real-time interaction. LLIA uses variable-length video generation technology to reduce the latency of initial video generation, and combines consistency model training strategies and model quantization techniques to significantly improve inference speed. LLIA supports using category labels to control the state of virtual avatars (such as speaking, listening, idle) and fine-grained control of facial expressions, providing users with a smooth and natural interactive experience.
Main Features of LLIA
- Real-time audio-driven portrait video generation: Generates corresponding portrait videos in real time based on input audio signals, achieving synchronization of speech with expressions and actions.
- Low-latency interaction: On high-performance GPUs, it can achieve high frame rates (e.g., 78 FPS at 384×384 resolution) and low latency (e.g., 140 ms) for video generation, suitable for real-time interaction scenarios.
- Multi-state switching: Supports controlling the state of virtual avatars based on category labels, such as speaking, listening, and idle states, allowing virtual avatars to react naturally according to the scenario.
- Facial expression control: Modifies the expressions of reference images based on portrait animation technology, thereby achieving fine-grained control of facial expressions in generated videos and enhancing the expressiveness of virtual avatars.
Technical Principles of LLIA
- Diffusion model framework: LLIA uses diffusion models as the foundational architecture, relying on powerful generation capabilities and high-fidelity output. Diffusion models generate images and videos by gradually removing noise.
- Variable-length video generation: LLIA introduces a dynamic training strategy that allows the model to generate video clips of different lengths during inference, reducing latency while maintaining video quality.
- Consistency Models: By introducing consistency models and discriminators, LLIA achieves high-quality video generation with fewer sampling steps, significantly accelerating inference speed.
- Model quantization and parallelization: Uses model quantization (such as INT8 quantization) and pipeline parallel technology to further optimize the model's inference performance and reduce computational resource requirements.
- Conditional input and control: Based on category labels and portrait animation technology, LLIA dynamically adjusts the state and expressions of virtual avatars according to the features of input audio, achieving natural interaction effects.
- High-quality dataset: LLIA is trained on a high-quality dataset of over 100 hours, including open-source data, web-collected data, and synthetic data, improving the model's performance capabilities in different scenarios.
LLIA Project Address
- Project official website：https://meigen-ai.github.io/llia/
- GitHub repository：https://github.com/MeiGen-AI/llia
- arXiv technical paper：https://arxiv.org/pdf/2506.05806
Application Scenarios of LLIA
- Virtual interviews: Generates virtual interviewers or candidates, enhancing the realism and interactivity of interviews based on real-time expression and action feedback.
- Mobile chatbots: Provides vivid virtual avatars for chatbots, generating expressions and actions in real time based on voice input, improving user interaction experience.
- Virtual customer service: Generate a virtual customer service representative that responds to customer voice in real time, using natural expressions and movements to improve customer satisfaction.
- Online Education: Generate a virtual teacher or teaching assistant that adjusts expressions and movements in real time based on teaching content and student feedback, enhancing teaching interactivity.
- Virtual Social Interaction: Generate a virtual avatar for users, controlling expressions and movements through voice to achieve a more realistic and natural social experience.