What Is ToonComposer
ToonComposer is a generative AI tool jointly launched by researchers from The Chinese University of Hong Kong, Tencent PCG ARC Lab, and Peking University. It can transform sketches into professional-grade animations in seconds. Based on generative post-keyframe technology, ToonComposer integrates the in-between frame generation and coloring steps in traditional animation production into an automated process. It requires only one sketch and one color reference frame to generate high-quality animation videos. The tool supports sparse sketch injection and regional control, allowing artists to accurately control the animation effects, greatly reducing manual workload, improving creative efficiency, and bringing revolutionary changes to animation production.
Main Features of ToonComposer
- Generative Post-Keyframe: Integrates the in-between frame generation and coloring steps in animation production into an automated process. It requires only one sketch and one color reference frame to produce a complete cartoon video, significantly reducing manual workload.
- Precise Sketch Keyframe Control: Artists guide the animation at any position on the timeline through sparse keyframe sketches, maintaining high precision and quality when handling complex motions.
- Regional Control: Users can selectively leave blank areas in the sketch and mark these areas with the brush tool. ToonComposer can intelligently fill the blank areas based on context or prompts, further reducing the artist's workload.
Technical Principles of ToonComposer
- Sparse Sketch Injection Mechanism: Based on positional encoding mapping, sketch frames are converted into tokens compatible with the model and injected into the latent space of the video generation model, ensuring that the sketch has temporal awareness during the generation process. At the same time, the position-aware residual module supports users in dynamically adjusting the sketch control strength during inference, enabling more flexible animation generation.
- Cartoon Adaptation Method: Based on the Spatial Low-Rank Adapter (SLRA), it only adjusts the model's spatial behavior without changing its temporal behavior, adapting modern video foundation models to the cartoon domain. This ensures that the generated animation visually conforms to the cartoon style while maintaining smooth motion effects.
- Regional Control Mechanism: During training, random masks are applied to sketch frames, and the model learns to generate reasonable content in these masked areas. During creation, users specify which areas need the model to automatically generate content, without the need to draw a complete sketch, further reducing workload and improving creative flexibility.
ToonComposer Project Address
- Project Official Website：https://lg-li.github.io/project/tooncomposer/
- GitHub Repository：https://github.com/TencentARC/ToonComposer
- HuggingFace Model Library：https://huggingface.co/TencentARC/ToonComposer
- arXiv Technical Paper：https://arxiv.org/pdf/2508.10881
- Online Experience Demo：https://huggingface.co/spaces/TencentARC/ToonComposer
Application Scenarios of ToonComposer
- Animation Production: Independent animation creators can quickly generate animation prototypes, reduce tedious in-between frame drawing and coloring, and focus their energy on creativity and storytelling.
- Game Development: Game developers can quickly generate animation assets in games, such as character motions and scene animations, reducing manual production costs and improving development efficiency.
- Education and Training: Used as a teaching tool in animation education to help students quickly understand the animation production process and enhance their interest in learning.
- Advertising and Marketing: Advertising agencies can quickly generate animated advertisements to meet different customer needs, saving time and cost when rapid iteration and modification of animation content are required.
- Film and Television Production: In the production of animated films and television series, generate preliminary animation clips to help directors and production teams quickly evaluate ideas and scene effects, reducing pre-production costs.