agentkit-multimedia-shopping

AgentKit Multimedia Xiaosheng Shopping Guide Digital Human Sales

Task Objective

This skill is used to: generate multimodal content for Xiaosheng Shopping Guide digital human sales videos based on the ByteDance agentkit-samples multimedia use case
Capabilities include:
- AI painting generation (Xiaosheng Shopping Guide character image, scene background)
- Speech synthesis (shopping guide voice, background music)
- Video generation (multimodal combination)
- Workflow orchestration (image→audio→video→finished video)
Trigger condition: the user needs to generate a Xiaosheng Shopping Guide digital human sales video and use the multimedia capabilities of agentkit-samples

Prerequisites

Dependency Description

This skill depends on the following Python packages:

Pillow>=10.0.0
requests>=2.28.0
numpy>=1.24.0

Environment Preparation

Install agentkit-samples (if needed)
Prepare API credentials (if third-party service calls are involved)
Prepare the fixed feature description of the Xiaosheng Shopping Guide character

Prerequisite Knowledge

Understand the basic functions of the ByteDance agentkit-samples multimedia use case
Understand the basic principles of AI painting, speech synthesis, and video generation
Understand 9:16 vertical video specifications

Operation Steps

Standard Process

1. Character Image Generation

Use AI painting to generate the Xiaosheng Shopping Guide character image:

Input Information：

Fixed character features (face shape, hairstyle, clothing, temperament)
Scene type (business scene)
Emotional tone (enthusiastic, professional, friendly)

Generation Process：

Read references/character-profile.md, obtain the fixed features of the Xiaosheng Shopping Guide character
Call scripts/generate_character.py to generate the character image
Crop to 9:16 ratio, resolution ≥1080×1920
Save the reference image (for use by InfiniteTalk)

Output: character reference image (9:16 vertical)

2. Scene Background Generation

Use AI painting to generate the scene background:

Input Information：

Scene type (business meeting room, study, negotiation room, lobby, office)
Emotional tone (cool tone, warm tone, neutral)
Light and shadow description (side light, front light, top light, backlight)

Generation Process：

Read the scene template and select the corresponding scene type
Call scripts/generate_scene.py to generate the scene background
Crop to 9:16 ratio, resolution ≥1080×1920
Save the scene reference image

Output: scene reference image (9:16 vertical)

3. Speech Synthesis

Use TTS to generate the shopping guide voice:

Input Information：

Script content (sales copy)
Voice type (speech rate, tone)
Emotional tone (enthusiastic, professional, friendly)

Generation Process：

Prepare the script content (in line with the Xiaosheng Shopping Guide persona)
Call scripts/generate_voice.py to synthesize speech
Export as 16kHz mono wav format
Save voice file (for use with InfiniteTalk)

Output: Voice file (16kHz mono wav)

4. Background music generation

Use a music generation tool to generate background music:

Input information：

Emotional tone (enthusiastic, professional, urgent, friendly)
Music style (orchestral, piano, strings)
Duration (5 seconds/scene)

Generation process：

Select music style based on emotional tone
Call scripts/generate_music.py to generate music
Export as 16kHz mono wav format
Save music file (for use with InfiniteTalk)

Output: Music file (16kHz mono wav)

5. Video generation

Use multimodal combination to generate video:

Input information：

Character reference image (9:16 vertical)
Scene reference image (9:16 vertical)
Voice file (16kHz mono wav)
Music file (16kHz mono wav)
Prompt (dedicated for InfiniteTalk)

Generation process：

Read the InfiniteTalk dedicated prompt (generated using the infinitetalk-shopping-avatar Skill)
Call scripts/generate_video.py to generate video
Generate 5 scenes of video (5 seconds each)
Concatenate the complete video (total duration 25 seconds)

Output: Complete video (25 seconds, 9:16 vertical)

Optional branches

When only the character image needs to be generated: execute step 1
When only the scene background needs to be generated: execute step 2
When only the voice needs to be generated: execute step 3
When only the background music needs to be generated: execute step 4
When only the video needs to be generated: execute steps 1-5

Resource index

Fixed character features: see references/character-profile.md(when to read: when generating the character image)
Scene template: see references/scene-templates.md(when to read: when generating the scene background)
Tool usage instructions: see references/tool-usage-guide.md(when to read: when using tools)
Example output: see assets/examples/sample-output.md(when to read: when referring to example output)

Notes

Character consistency: All generated character images must strictly follow the fixed character features
Resolution matching: All images must be 9:16 vertical, with resolution ≥1080×1920
Audio format: All audio must be in 16kHz mono wav format
Emotion matching: Voice and music must match the scene emotion
Workflow order: Strictly follow the order of image → audio → video for generation
Coordination with InfiniteTalk: Use the infinitetalk-shopping-avatar Skill to generate prompts

Usage examples

Example 1: Generate a complete product promotion video

Function description: The complete process for generating a digital human product promotion video for the Xiaosheng shopping guide

Execution method：

Generate character image (call generate_character.py)
Generate scene background (call generate_scene.py)
Synthesize the sales assistant's voice (call generate_voice.py)
Generate background music (call generate_music.py)
Generate video (call generate_video.py, combined with the InfiniteTalk prompt)

Key parameters：

Fixed character features: strictly follow character-profile.md
Scene type: business scene
Emotional tone: enthusiastic and professional
Resolution: 1080×1920 (9:16 vertical screen)
Audio format: 16kHz mono wav

Output: complete product promotion video (25 seconds, 9:16 vertical screen)

Example 2: Generate only the character reference image

Function description: generate only the Xiaosheng sales assistant character reference image

Execution method: call generate_character.py

Key parameters：

Fixed character features: strictly follow character-profile.md
Emotional tone: enthusiastic and professional
Resolution: 1080×1920 (9:16 vertical screen)

Output: character reference image (9:16 vertical screen)

Workflow summary

角色固定特征描述
    ↓
AI绘画生成（角色形象+场景背景）
    ↓
语音合成（导购员语音+背景音乐）
    ↓
InfiniteTalk提示词生成（使用infinitetalk-shopping-avatar Skill）
    ↓
多模态视频生成（图像+音频+提示词）
    ↓
完整带货视频（25秒，9:16竖屏）

Collaboration with InfiniteTalk

This skill works in collaboration with theinfinitetalk-shopping-avatar Skill:

This skill: generate multimodal content (images, audio)
infinitetalk-shopping-avatar Skill: generate InfiniteTalk-specific prompts
InfiniteTalk: use multimodal content and prompts to generate video

Collaboration process:

本技能生成角色参考图 → infinitetalk-shopping-avatar生成提示词 → InfiniteTalk生成视频
本技能生成语音文件 → chinese-wav2vec2-base编码 → InfiniteTalk使用
本技能生成音乐文件 → chinese-wav2vec2-base编码 → InfiniteTalk使用

About this skill

Task Objective

Prerequisites

Dependency Description

Environment Preparation

Prerequisite Knowledge

Operation Steps

Standard Process

1. Character Image Generation

2. Scene Background Generation

3. Speech Synthesis

4. Background music generation

5. Video generation

Optional branches

Resource index

Notes

Usage examples

Example 1: Generate a complete product promotion video

Example 2: Generate only the character reference image

Workflow summary

Collaboration with InfiniteTalk