What is Textoon
Textoon is an innovative project launched by Alibaba Group's Tongyi Lab, the first method capable of generating Live2D-format 2D cartoon characters based on text prompts. Based on advanced language and vision models, it can generate diverse and interactive 2D characters within one minute. The generated characters support animation effects, are suitable for HTML5 rendering, and can be widely applied in fields such as film and television, games, social media, and advertising.
Main Features of Textoon
- Text Parsing: Textoon uses a fine-tuned large language model (LLM) to extract detailed information from complex text, such as hairstyle, eye color, clothing type, etc.
- Controllable Appearance Generation: Through text-to-image models (such as SDXL), Textoon can generate the appearance of characters based on user input and ensure that the generated results match the user's description.
- Editing and Repair: Users can edit the generated characters, adjust details, or repair unsatisfactory parts.
- Animation Enhancement: Textoon integrates ARKit's facial expression parameters, significantly improving the animation expressiveness of characters.
- Multilingual Support: Textoon supports Chinese and English prompts, meeting the language needs of different users.
Technical Principles of Textoon
- Text Parsing and Intent Understanding: Textoon is based on a fine-tuned large language model (LLM) and can extract detailed information from complex text descriptions, such as hairstyle, hair color, eye color, clothing type, etc. The accurate text parsing capability enables the system to understand the user's input description and transform it into specific visual features.
- Live2D Technology Integration: The generated characters are output in Live2D format, which supports efficient rendering performance and flexible animation expression. Live2D technology uses multi-layered 2D drawing and bone binding to give characters rich expression and motion capabilities. Textoon also integrates ARKit capabilities, further enhancing the animation expressiveness of characters.
- Efficient Rendering and Compatibility: The generated Live2D models have efficient rendering performance, are suitable for HTML5 rendering, and are suitable for use in mobile devices and web applications.
Project Address of Textoon
- Project Official Website：https://human3daigc.github.io/Textoon_webpage/
- arXiv Technical Paper：https://arxiv.org/pdf/2501.10020
Application Scenarios of Textoon
- Creative Design: Textoon can help designers quickly generate character prototypes, saving design time and cost. Through simple text descriptions, users can generate richly detailed 2D cartoon characters in a short time and edit and adjust them.
- Game Development: In game development, developers can use Textoon to quickly generate character prototypes for game concept design or directly apply them to character animation in games.
- Anime Production: Textoon can generate high-quality 2D cartoon characters based on detailed text descriptions, suitable for character design and animation production in anime production. The generated characters can be directly used in the production of animated videos or serve as references for animators.
- Literary Creation: For literary creators, Textoon can visualize the character images in textual descriptions, helping authors better conceive character appearance and personality traits and enhancing the visual effect of creation.
- Education and Training: Textoon can be used to create instructional videos or interactive learning materials. By generating vivid 2D cartoon characters, it can provide students with a more intuitive and interesting learning experience, for example, by creating virtual teachers or animated explanations.