EchoMimicV2 is an open-source AI digital human model launched by the Terminal Technology Department of Ant Group (Alipay, Ant Group). It belongs to the EchoMimic series and is positioned as "towards striking, simplified, and half-body portrait animation." According to official materials, EchoMimicV2 uses a reference image, an audio clip, and a series of hand poses to generate high-quality animated videos and ensure consistency between the audio content and half-body movements.
In terms of use, EchoMimicV2 focuses on half-body portrait animation generation, with the goal of achieving expressive half-body animation while simplifying unnecessary control conditions. Unlike portrait animation work that typically relies on audio, pose, or motion map conditions, EchoMimicV2 attempts to reduce additional control conditions, cumbersome condition injection modules, and restrictions on driving the head region.
In terms of main capabilities, this method proposes a novel audio-pose dynamic coordination strategy, including pose sampling and audio diffusion, to enhance half-body details, facial and gesture expressiveness, while reducing condition redundancy. To compensate for the scarcity of half-body data, it adopts Head Partial Attention to seamlessly integrate headshot data into the training framework, and this mechanism can be omitted during inference, thereby facilitating animation generation. In addition, it designs a Phase-specific Denoising Loss to guide the motion, details, and low-level quality of the animation at different stages. This work also proposes a new benchmark for evaluating the effectiveness of half-body portrait animation.
In terms of usage, official materials show related examples, including Chinese driving audio, English driving audio, comparisons with pose-driven algorithms, comparisons with audio-driven algorithms, and reference images generated by FLUX. The official website address is https://antgroup.github.io/ai/echomimic_v2/ . Regarding specific installation commands, code invocation methods, dependency environments, model weight acquisition channels, etc., official materials do not currently provide them.
In terms of applicable scenarios, based on the official description, EchoMimicV2 can be applied to scenarios requiring half-body portrait animation generation, such as digital human video production driven by audio and hand poses.
In terms of limitations, official materials mention that existing methods often face practical challenges due to additional control conditions, cumbersome condition injection modules, or being limited to driving the head region; EchoMimicV2 is proposed precisely to address these issues. Official materials state that extensive experiments and analysis show that EchoMimicV2 outperforms existing methods in both quantitative and qualitative evaluations. However, regarding its specific performance metric values, supported language range, specific requirements for input images and audio, inference speed, hardware requirements, etc., official materials do not currently provide them.
