video-recreation

video-recreation

Task Objective

This Skill is used for: video re-creation, the complete process from reverse-engineering analysis of the original video to synthesis of a new video
Capabilities include: video analysis, material generation (images/audio/voiceover/subtitles), video synthesis, file download
Trigger conditions: when the user requests "re-created video", "reverse-engineered video", "video remake", "create based on a reference video", etc.

Prerequisites

Environment dependencies:

opencv-python>=4.8.0
pillow>=10.0.0
moviepy>=1.0.3
numpy>=1.24.0
requests>=2.28.0
edge-tts>=6.1.0

Edge-TTS installation:
```
pip install edge-tts
```
Suno API (optional):
- Used to generate realistic background music
- Three configuration methods:
  1. Developer mode: The skill has a preset API Key, ready to use out of the box
  2. User mode: Set the environment variable export SUNO_API_KEY=your_api_key
  3. Command-line mode: Specify at execution time --suno-api-key your_api_key
- Configuration priority: command-line parameters > environment variables > skill credentials
- Placeholder mode: automatically use the placeholder implementation when no API Key is configured
- Official website: https://suno.com
Environment variables:
- COZE_BOT_ID: Coze Bot ID (default: 7572557757883383858)
- COZE_API_KEY: Coze API Key (must be configured)

Output directory structure:

./output/
  ├── frames/          # 视频关键帧
  ├── analysis.json    # 反推分析结果
  ├── images/          # 生成的图片素材
  ├── audio/           # 音效和背景音乐
  ├── voice/           # 配音音频
  ├── subtitles/       # 字幕文件
  └── final.mp4        # 最终合成视频

Operation steps

Phase 1: Video reverse-engineering analysis

Extract video keyframes
- Call scripts/video_frame_extractor.py to extract keyframes
- Parameters: --input <原视频路径> --output ./output/frames --interval 2
- Output: sequence images to ./output/frames/
Visual analysis
- Call scripts/coze_bot_client.py to analyze keyframes
- Agent describes the analysis requirements: "Analyze these video frames and extract: 1. Visual style 2. Color tone characteristics 3. Composition method 4. Rhythm pattern"
- Parameters: --message "<分析提示>" --image_path <关键帧路径>
- Output: save the analysis results to ./output/analysis.json
Generate a creation plan
- The agent generates a re-creation plan based on the analysis results:
  - New video theme
  - Visual style adjustments
  - Script outline
  - Material requirements list

Phase 2: Material generation

Generate image materials
- The agent generates keyframe prompts based on the script
- Call scripts/image_generator.py to generate images
- Parameters: --prompt "<提示词>" --output ./output/images/frame_xxxx.png
- Important: image generation should be created by the agent; the script is only responsible for technical implementation
- Output: images to ./output/images/
Generate sound effects and background music
- The agent describes the sound effect requirements: "Generate sound effects based on the scene: 1. Transition sound effects 2. Background music style 3. Ambient sound"
- Call scripts/sound_generator.py to generate sound effects and background music
- Parameters: --type sound --input <音效配置JSON> --output ./output/audio (generate sound effects)
- Parameters: --type music --input <音乐配置JSON> --output ./output/audio (generate background music)
- Parameters: --type both --input <完整配置JSON> --output ./output/audio (generate both at the same time)
- API Key configuration(optional):
  - Not configured: Automatically uses placeholder implementation (ready to use out of the box, no configuration required)
  - Environment variable: export SUNO_API_KEY=your_api_key
  - Command-line parameter: --suno-api-key your_api_key
  - Force placeholder: --use-placeholder (use placeholder even if an API Key is available)
- Output: sound effects to ./output/audio/sound_effects/, background music to ./output/audio/background_music/
- Mode Description:
  - Developer Mode: The skill has a preset API Key, use it directly
  - User Mode: Users configure their own API Key
  - Placeholder Mode: Automatically falls back when no API Key is configured
Generate Voiceover
- The agent acts as a voiceover artist to create narration scripts:
  - Analyze the rhythm of the original video
  - Create narration that fits the visuals
  - Choose suitable voices (20+ voices including Chinese/English/Japanese, etc.)
  - Adjust speech rate, pitch, and volume parameters
- Call scripts/voice_generator.py to synthesize voiceover (based on Edge-TTS)
- Parameter: --input <旁白脚本JSON> --output ./output/voice
- Parameter: --list-voices (list all available voices)
- Output: voiceover file to ./output/voice/
- Voice Examples:
  - Chinese female voices: zh-CN-XiaoxiaoNeural (lively), zh-CN-XiaohanNeural (intellectual), zh-CN-XiaomengNeural (gentle)
  - Chinese male voices: zh-CN-YunyangNeural (steady), zh-CN-YunjianNeural (deep), zh-CN-YunxiNeural (lively)
  - English female voices: en-US-JennyNeural (American), en-GB-SoniaNeural (British)
Generate Subtitles
- The agent creates subtitle content, ensuring:
  - Concise and impactful text
  - Synchronized with the visuals
  - Matches the video rhythm
- Call scripts/subtitle_generator.py to generate the subtitle file
- Parameter: --input <字幕数据JSON> --output ./output/subtitles
- Output: SRT subtitles to ./output/subtitles/

Stage 3: Video Composition

Composite Final Video
- Call scripts/video_compositor.py to compose the video
- Parameter: --images ./output/images --audio ./output/audio --voice ./output/voice --subtitles ./output/subtitles --output ./output/final.mp4
- Output: final video ./output/final.mp4
File Download
- Start an HTTP server for download
- Call scripts/file_server.py
- Parameter: --port 8080 --directory ./output
- Output: download link http://localhost:8080/final.mp4

Error Handling and Resumable Processing

Retry Mechanism: All API calls are configured with a maximum number of retries (2-3 times) to avoid unlimited Token consumption
Error Log: Errors are automatically logged to ./output/error_log.json, which can be used for troubleshooting
Resumable Processing:
- Check ./output/error_log.json to confirm the failed step
- Re-execute from the failed step; already generated assets can be reused
- For example: only regenerate the failed audio, without repeating existing images
Retry Limits:
- Coze Bot API calls: retry up to 3 times, with a 1-second interval each time
- Image generation: retry up to 2 times, with a 0.5-second interval each time
- Audio generation: retry up to 2 times, with a 0.5-second interval each time
- Sound effects and background music: retry up to 2 times, with a 0.5-second interval each time

Resource Index

Video Processing: see scripts/video_frame_extractor.py(Extract key frames)
Visual analysis: see scripts/coze_bot_client.py(Call Coze Bot API)
Image generation: see scripts/image_generator.py(Generate image assets)
Audio generation: see scripts/audio_generator.py(Generate narration/voiceover)
Sound effects and music: see scripts/sound_generator.py(Generate ambient sound effects and background music, integrate Suno API)
Voiceover synthesis: see scripts/voice_generator.py(Synthesize narration, based on Edge-TTS)
Subtitle generation: see scripts/subtitle_generator.py(Generate subtitles)
Video synthesis: see scripts/video_compositor.py(Synthesize final video)
File service: see scripts/file_server.py(HTTP download server)
Error handling: see scripts/error_handler.py(Retry and error logs)
Creative guide: see references/recreation-guide.md(Video recreation methodology)
Prompt templates: see references/prompt-templates.md(Analysis prompt examples)
Suno API guide: see references/suno-api-guide.md(Suno API usage instructions)

Notes

Agent responsibilities: content creation (script, narration, subtitles, image prompts) is done by the agent, and the script handles technical processing
Retry limit: to avoid infinite retries consuming Tokens, a reasonable number of retries has been configured
Error logs: when encountering an error, check ./output/error_log.json, and resume from the failed step
Resume from breakpoint: when executed repeatedly, existing assets will be reused without needing to be regenerated
Coze Bot API: visual analysis depends on the Coze Bot published by the user, and an API Key must be configured
File paths: all outputs use relative paths ./output/, ensuring they can be accessed correctly during download

Usage examples

Example 1: Complete recreation workflow

# 1. 提取关键帧
python scripts/video_frame_extractor.py \
  --input original_video.mp4 \
  --output ./output/frames \
  --interval 2

# 2. 视觉分析(智能体描述分析需求)
python scripts/coze_bot_client.py \
  --message "分析这些视频帧,提取:画面风格、色调特征、构图方式、节奏模式" \
  --image_path ./output/frames/frame_0001.jpg

# 3. 生成图片素材(智能体创作提示词)
python scripts/image_generator.py \
  --prompt "现代科技风格,蓝色调,未来城市景观" \
  --output ./output/images/frame_0001.png

# 4. 生成音效和背景音乐

# 方式1: 使用技能预置的 API Key(开箱即用)
python scripts/sound_generator.py \
  --type both \
  --input audio_config.json \
  --output ./output/audio

# 方式2: 使用自己的 API Key
export SUNO_API_KEY=your_api_key
python scripts/sound_generator.py \
  --type both \
  --input audio_config.json \
  --output ./output/audio

# 方式3: 命令行指定 API Key
python scripts/sound_generator.py \
  --type both \
  --input audio_config.json \
  --output ./output/audio \
  --suno-api-key your_api_key

# 方式4: 强制使用占位实现(不调用 API)
python scripts/sound_generator.py \
  --type both \
  --input audio_config.json \
  --output ./output/audio \
  --use-placeholder

# 其中audio_config.json示例:
{
  "sound_effects": [
    {"name": "transition_01", "type": "transition", "duration": 2.0, "description": "转场音效"},
    {"name": "impact_01", "type": "impact", "duration": 0.5, "description": "冲击音效"}
  ],
  "background_music": {
    "name": "background",
    "style": "calm",
    "duration": 60.0,
    "tempo": 90,
    "mood": "neutral"
  }
}

# API Key 配置优先级: 命令行参数 > 环境变量 > 技能凭证 > 占位实现

# 5. 生成配音(Edge-TTS)
# 查看可用音色
python scripts/voice_generator.py --list-voices

# 生成配音
python scripts/voice_generator.py \
  --input narration.json \
  --output ./output/voice

# 其中narration.json示例:
{
  "segments": [
    {
      "segment_id": "S01",
      "text": "欢迎来到这个美丽的世界",
      "voice": "zh-CN-XiaomengNeural",
      "rate": "-10%",
      "pitch": "+0Hz",
      "volume": "+0%"
    },
    {
      "segment_id": "S02",
      "text": "让我们开始这段奇妙的旅程",
      "voice": "zh-CN-YunyangNeural",
      "rate": "-5%",
      "pitch": "-2Hz",
      "volume": "+5%"
    }
  ]
}

# 6. 生成字幕
python scripts/subtitle_generator.py \
  --input subtitle_data.json \
  --output ./output/subtitles

# 7. 合成视频
python scripts/video_compositor.py \
  --images ./output/images \
  --audio ./output/audio \
  --voice ./output/voice \
  --subtitles ./output/subtitles \
  --output ./output/final.mp4

# 8. 启动下载服务器
python scripts/file_server.py \
  --port 8080 \
  --directory ./output

Example 2: Generate sound effects separately

python scripts/sound_generator.py \
  --type sound \
  --input sound_effects.json \
  --output ./output/audio

Example 3: Generate background music separately

python scripts/sound_generator.py \
  --type music \
  --input background_music.json \
  --output ./output/audio

Example 4: Resume from breakpoint

# 检查错误日志
cat ./output/error_log.json

# 从失败步骤重新执行(例如仅重新生成失败的音效)
python scripts/sound_generator.py \
  --type sound \
  --input sound_effects.json \
  --output ./output/audio

Agent role division

Visual analyst: analyze the original video and extract features such as style, color tone, and composition
Script planner: create a new video script and outline based on the analysis results
Prompt designer: create precise prompts for image generation tools
Voiceover artist: create the narration script and determine voice tone, rhythm, and emotion
Sound effects artist: design the sound effects plan and determine sound effect type, duration, and position
Musician: choose the background music style and determine the rhythm and emotional tone
Subtitle artist: create subtitle content and ensure it is synchronized with the visuals
Technical coordinator: call scripts to complete technical processing, manage file paths, and start the download service

About this skill

Task Objective

Prerequisites

Operation steps

Phase 1: Video reverse-engineering analysis

Phase 2: Material generation

Stage 3: Video Composition

Error Handling and Resumable Processing

Resource Index

Notes

Usage examples

Example 1: Complete recreation workflow

Example 2: Generate sound effects separately

Example 3: Generate background music separately

Example 4: Resume from breakpoint

Agent role division