What is Taobao Xingchen
Taobao Xingchen is an e-commerce video generation large model launched by Alimama, designed specifically for Taobao and Tmall merchants. It can generate high-quality videos in one click based on images or text instructions input by merchants. Its core advantage lies in a better understanding of product presentation, with the ability to accurately identify and apply presentation techniques suitable for different products; it has strong multilingual semantic-following capabilities, ensuring that generated videos are highly consistent with user intent; it follows physical and motion laws, making the visuals smooth and natural; and it can stably maintain the clarity and completeness of people, products, and decorative elements. Merchants can use it to add vivid videos to product detail pages and enhance product appeal. Currently, this tool has been made available for use by Taobao and Tmall merchants.
Main features of Taobao Xingchen
- A better understanding of product presentation techniques: Based on rich e-commerce data and design experience, it automatically identifies and applies the presentation techniques most suitable for different categories of products. From the precise presentation of product details to the unified coordination of the overall style, it ensures that every product can be displayed in the best way.
- Stronger multilingual semantic-following capability: It deeply understands reference images and multilingual instructions (Chinese/English), accurately captures key action information in text instructions, and ensures that generated videos are highly consistent with user intent.
- Reasonable physical and motion laws: It strictly follows physical laws and motion norms, ensuring smooth and natural visuals, avoiding unrealistic visual effects, and improving the realism and professionalism of videos.
- Stable preservation of people, products, and decorations: Whether faces, products, or decorative elements such as text and stickers, they always remain clear, complete, and free of jitter or distortion, improving visual consistency.
Technical principles of Taobao Xingchen
- Based on the Unet Diffusion Model approach: The Taobao Xingchen video generation large model was developed on the basis of the Unet Diffusion Model approach. The Unet Diffusion Model is a generative model that generates data by gradually removing noise. It can learn the complex distribution of data and generate high-quality video content.
- Self-developed 3D VAE: To better handle three-dimensional information and spatial structure in videos, Taobao Xingchen developed its own 3D VAE (Variational Autoencoder). The 3D VAE can perform three-dimensional modeling and encoding of objects, scenes, and more in videos, mapping complex three-dimensional information into a low-dimensional latent space, and then reconstructing high-quality video frames from the latent space through a decoder, making the generated videos more realistic and natural in spatial structure and object form.
- A data flywheel oriented toward generation tasks: A data flywheel oriented toward video generation tasks was built. By continuously collecting, annotating, and utilizing massive amounts of e-commerce data, it provides rich materials and supervisory information for model training. The data covers images, videos, text descriptions, and more for various products, enabling the model to learn the presentation techniques, style characteristics, and correspondence with user intent for different products, so that when generating videos it can more accurately meet the needs of e-commerce scenarios.
- Multi-model collaboration: The Taobao Xingchen video generation large model includes multiple sub-models such as Tbstar-T2V (text-to-video), Tbstar-I2V (image-to-video), and Tbstar-V2V (video-to-video). The sub-models collaborate with one another to jointly complete video generation tasks. This further enriches the functions and effects of video generation.
How to use Taobao Xingchen
- Visit the platform: Log in to Taobao Xingchen'sOfficial Website。
- Select a Function Module: Select the "Image to Video" function module.
- Input Material: Upload a high-quality clothing model display image.
- Generate Video: Click the generate button and wait for the video to be generated.
- Video Editing: The generated video will automatically be displayed on the page, and you can perform simple edits, such as adding a brand logo, adjusting colors, etc.
- Download and Use: After editing is complete, download the video and add it to the product detail page or use it for social media promotion.
- Usage Restrictions: Currently, Taobao Xingchen is only open to Taobao and Tmall merchants, and ordinary users cannot use it for the time being. Merchants need to ensure that their accounts have the corresponding permissions.
Application Scenarios of Taobao Xingchen
- Product Main Image Video: A clothing merchant uploaded a high-quality model display image to Taobao Xingchen and generated a 5-second video main image for the product detail page, significantly increasing the product's click-through rate and conversion rate.
- Eye-Catching Selling Point Video: An electronics merchant uploaded multiple product images and generated multiple video clips. By using the template video method to add transitions, special effects, and text, they created a selling point video showcasing the product's features and advantages for social media promotion.
- Recommendation Content Video: A beauty merchant uploaded multiple beauty product images and generated multiple video clips. By using the video editing method to add narration and subtitles, they created a recommendation video and published it on platforms such as Xiaohongshu to stimulate consumers' desire to purchase.
- Virtual Try-On Video: A clothing merchant uploaded a user's virtual try-on image and generated a virtual try-on video to simulate the real wearing effect of the clothing, helping consumers better understand the product and reduce purchase concerns.