SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsvideo-creation-suite

video-creation-suite

Design & creation

View repository

Updated 10-16 09:53

About this skill

Video Creation Kit

Task Objective

  • This Skill is used to: provide complete video creation capabilities, supporting three creation modes
  • Capabilities include:
    • Mode 1: Original Creation: 11 agents collaborate on original video creation (The Three-Body Problem IP creation, general original creation)
    • Mode 2: Video Secondary Creation: A complete secondary creation workflow from reverse-engineering the original video to synthesizing a new video
    • Mode 3: Video Analysis: Video frame extraction, visual analysis, content extraction
    • Technical Capabilities: Voiceover generation, sound effect generation, background music generation, video synthesis
  • Trigger conditions: The user needs to perform video creation (original creation/secondary creation/analysis)

Three Creation Modes

Mode 1: Original Creation

  • Applicable Scenarios: Create an original video from scratch
  • Sub-modes:
    • The Three-Body Problem IP creation (specifically for The Three-Body Problem IP)
    • General multi-agent collaboration (division of labor among 11 agents)
  • Workflow: Topic planning → Visual design → Material generation → Video synthesis → Quality inspection

Mode 2: Video Secondary Creation

  • Applicable Scenarios: Perform secondary creation based on an existing video
  • Workflow: Video reverse-engineering analysis → Material generation → Video synthesis → File download
  • Capabilities: Video frame extraction, visual analysis, image generation, voiceover, sound effects, music, synthesis

Mode 3: Video Analysis

  • Applicable Scenarios: Analyze video content, extract storyboard references, generate creation prompts
  • Workflow: Video frame extraction → Visual analysis → Structured output
  • Capabilities: Frame extraction, visual model invocation, prompt generation

Prerequisites

Environment Dependencies

opencv-python>=4.8.0
pillow>=10.0.0
moviepy>=1.0.3
numpy>=1.24.0
requests>=2.28.0
edge-tts>=6.1.0

API Configuration (Optional)

  • Coze Bot API: Visual analysis
    • Environment variable: COZE_BOT_ID, COZE_API_KEY
  • Edge-TTS: Voiceover generation
    • Install: pip install edge-tts
  • Suno API: Background music generation
    • Environment variable: SUNO_API_KEY
    • Three modes: developer, user, placeholder

Operation Steps

Mode Selection

The agent automatically selects the mode based on user input:

Determine Original Creation Mode

  • User mentions "create a The Three-Body Problem video," "The Three-Body Problem IP creation" → The Three-Body Problem IP creation
  • User mentions "multi-agent collaboration," "original video," "create from scratch" → General multi-agent collaboration

Determine Video Secondary Creation Mode

  • User mentions "secondary creation video," "reverse-engineer video," "video remake," "create based on a reference video" → Video secondary creation

Determine Video Analysis Mode

  • User mentions "analyze video," "extract storyboard," "generate prompts" → Video analysis

Mode 1: Original Creation

Sub-mode 1: The Three-Body Problem IP creation

Phase 1: Pre-production Preparation

  1. Source Material Research and Setting Extraction

    • Read references/three-body-settings.md
    • Extract the core settings of The Three-Body Problem 2: Deterrence Era
    • Output: Timeline anchor table, character behavior compliance checklist

  2. Plot Structure Design

    • Build a "three-line parallel" narrative framework
    • Design 3 core conflict points and 1 emotional climax
    • Output: Three-line parallel narrative storyboard outline, emotional curve chart

  3. Dialogue Writing

    • Write dialogue based on character personalities
    • Output: Character dialogue script (including tone/pause markers)

Phase 2: Visual Production

  1. Visual Style Specification Development

  2. Scene Design

    • Design 3 core scenes
    • Output: Scene concept art prompts, scene element checklist

  3. Character Consistency Modeling

    • Luo Ji image anchoring
    • Output: Luo Ji standard image anchor atlas, character state comparison table

  4. Storyboard Script Refinement

    • Convert the plot outline into professional storyboards
    • Output: Film storyboard script (including time codes), rendering metadata

Phase 3: Audio and Editing

  1. Asset Generation

    • Voiceover Generation: Call scripts/voice_generator.py to generate character voiceover
      • Parameters: --input <旁白数据JSON> --output ./output/audio/voice
    • Sound Effect Generation: Call scripts/sound_generator.py --type sound to generate sound effects
      • Parameters: --input <音效配置JSON> --output ./output/audio/sound_effects
    • Background Music Generation: Call scripts/sound_generator.py --type music to generate background music
      • Parameters: --input <音乐配置JSON> --output ./output/audio/background_music
    • Video Compositing: Call scripts/video_compositor.py to composite the final video
      • Parameters: --images <图片目录> --audio <音频目录> --subtitles <字幕文件> --output <输出路径>

  2. Quality Control

    • Call scripts/quality_checker.py to detect technical indicators
    • Source material compliance detection
    • Bilibili adaptation optimization

Sub-mode 2: General Multi-Agent Collaboration

Division of labor among 11 agents

  1. Copywriting
  2. Story Planning
  3. Script Writing
  4. Storyboard Director
  5. Storyboard Artist
  6. Subtitle Artist
  7. Voiceover Artist
  8. Sound Effects Artist
  9. Video Engineer
  10. Quality Inspection
  11. Data Feedback

5-Phase Collaborative Workflow

  1. Requirement Intake
  2. Content Creation
  3. Image Generation
  4. Audio and Subtitles
  5. Video Compositing
  6. Full-Process Quality Control
  7. Data Iteration

Mode 2: Video Secondary Creation

Phase 1: Video Reverse-Engineering Analysis

  1. Extract Video Keyframes

    • Call scripts/video_frame_extractor.py to extract keyframes
    • Parameters: --input <原视频路径> --output ./output/frames --interval 2
    • Output: Sequence images to ./output/frames/

  2. Visual Analysis

    • Call scripts/coze_bot_client.py to analyze keyframes
    • Agent description analysis requirements: "Analyze these video frames and extract: visual style, color tone characteristics, composition method, rhythm pattern"
    • Output: Save the analysis results to ./output/analysis.json

  3. Generate creation plan

    • The agent generates a derivative creation plan based on the analysis results
    • Output: new video theme, visual style adjustments, script outline, material requirements list

Phase 2: Material generation

  1. Generate image materials

    • The agent generates keyframe prompts based on the script
    • Call scripts/image_generator.py to generate images
    • Output: images to ./output/images/

  2. Generate voiceover

    • The agent acts as a voice actor and creates the narration script
    • Call scripts/voice_generator.py to synthesize voiceover (based on Edge-TTS)
    • Parameters: --input <旁白脚本JSON> --output ./output/voice
    • Output: voiceover file to ./output/voice/

  3. Generate sound effects and background music

    • Call scripts/sound_generator.py --type both to generate sound effects and background music
    • Parameters: --input <完整配置JSON> --output ./output/audio
    • Output: sound effects to ./output/audio/sound_effects/, background music to ./output/audio/background_music/

  4. Generate subtitles

    • The agent creates subtitle content
    • Call scripts/subtitle_generator.py to generate subtitle file
    • Parameters: --input <字幕数据JSON> --output ./output/subtitles
    • Output: SRT subtitles to ./output/subtitles/

Phase 3: Video composition

  1. Compose final video

    • Call scripts/video_compositor.py to compose video
    • Parameters: --images ./output/images --audio ./output/audio --voice ./output/voice --subtitles ./output/subtitles --output ./output/final.mp4
    • Output: final video ./output/final.mp4

  2. File download

    • Call scripts/file_server.py to start HTTP server
    • Parameters: --port 8080 --directory ./output
    • Output: download link http://localhost:8080/final.mp4

Mode 3: Video analysis

Analysis workflow

  1. Video frame extraction

    • Call scripts/video_frame_extractor.py to extract keyframes
    • Parameters: --input <视频路径> --output <输出目录> --interval <间隔秒数>

  2. Visual analysis

    • Call scripts/coze_bot_client.py to analyze each frame's content
    • Parameters: --message "<分析提示>" --image_path <图片路径>
    • Output: structured analysis results (JSON format)

  3. Batch processing

    • Automatically iterate through all keyframes
    • Batch call the visual model
    • Aggregate analysis results

Resource index

Technical scripts

Reference documents

Resource files

Notes

  • Automatic mode selection: The agent automatically selects the appropriate creation mode based on user input
  • Script sharing: All modes share the same set of technical scripts to avoid duplication
  • API configuration: All APIs are configured through environment variables, supporting multiple modes
  • Error handling: All scripts include error handling and retry mechanisms
  • Fault-tolerant degradation: Automatically degrades to a placeholder implementation when the API is unavailable

Usage examples

Example 1: 《The Three-Body Problem》IP creation

用户: "帮我创作一个《三体》威慑纪元的8分钟视频"

智能体执行:
1. 自动选择模式一: 《三体》IP创作
2. 按照前期筹备 → 视觉制作 → 音频与剪辑 → 质量管控流程
3. 调用相应脚本生成配音、音效、背景音乐
4. 合成最终视频

Example 2: Video secondary creation

用户: "这个视频帮我二创一下，换个风格"

智能体执行:
1. 自动选择模式二: 视频二创
2. 提取原视频关键帧
3. 视觉分析，提取风格特征
4. 生成新素材（图片、配音、音效、音乐）
5. 合成新视频

Example 3: Video analysis

用户: "分析这个视频，提取分镜和提示词"

智能体执行:
1. 自动选择模式三: 视频分析
2. 抽取视频关键帧
3. 分析每帧内容
4. 输出结构化分析结果

API configuration instructions

Suno API three modes

  • Developer mode: Skill preset API Key, ready to use out of the box
  • User mode: Users configure their own API Key
  • Placeholder mode: Automatic degradation, completely free

Edge-TTS voice list

  • Chinese: zh-CN-XiaoxiaoNeural(lively), zh-CN-XiaomengNeural(gentle), zh-CN-YunyangNeural(steady)
  • English: en-US-JennyNeural(American), en-GB-SoniaNeural(British)

Coze Bot API

  • Used for visual analysis
  • Environment variables: COZE_BOT_ID, COZE_API_KEY