manga-workflow

Video Workflow

Complete end-to-end workflow that converts a novel into a video.

Content Mode

The system supports two content modes, switched via the project.json field in content_mode :

Mode	content_mode	Aspect Ratio	Default Duration	Agent
Narration + visuals (default)	`narration`	9:16 portrait	4 seconds	novel-to-narration-script
Episodic animation	`drama`	16:9 landscape	8 seconds	novel-to-storyboard-script

Workflow Decision Tree

开始
  │
  ├─ 新项目？ ──────────────────┐
  │                             │
  │   1. 创建项目目录            │
  │   2. 将小说复制到 source/    │
  │   3. 创建 project.json       │
  │   4. 选择 content_mode       │
  │   └─────────────────────────┘
  │
  ├─ 检查项目状态 (project.json)
  │   │
  │   ├─ 没有剧本？
  │   │   ├─ narration 模式 → novel-to-narration-script agent（3 步）
  │   │   └─ drama 模式 → novel-to-storyboard-script agent（4 步）
  │   │   └─► 审核 ──┐
  │   │
  │   ├─ 没有确认线索？ ───► 确认 clues 列表 ──► 更新 project.json ──┐
  │   │
  │   ├─ 没有人物设计？ ────► /generate-characters ──► 审核 ──┐
  │   │
  │   ├─ 没有线索设计？ ────► /generate-clues ──► 审核 ──┐
  │   │
  │   ├─ 没有分镜图？ ──────► /generate-storyboard ──► 审核 ──┐
  │   │
  │   ├─ 没有视频？ ────────► /generate-video ──► 审核 ──┐
  │   │
  │   └─ 准备合成？ ───────► /compose-video ──► 完成！

Project Overview: After the source file is uploaded, the system automatically analyzes the novel content and generates a project overview (story synopsis, genre type, core themes, worldbuilding settings). The overview information is saved to the project.json field in overview , for reference by subsequent Agents.

Stage 1: Project Setup

New Project：

Ask for the project name
Create projects/{名称}/ and subdirectories (including clues/、drafts/）
Create project.json initial file
Ask for the content mode：narration(default) or drama
Ask the user to put the novel text into source/
Automatically generate the project overview after upload（synopsis、genre、theme、world_setting）

Existing Project：

List the projects in projects/ project in
Display the project status in project.json project status in
Continue from the last unfinished stage

Stage 2: Script Creation

Based on content_mode , call different Agents (using the Opus model):

Narration + visuals mode (narration)

Use the Task tool to call novel-to-narration-script agent：

Core principles: preserve the original text, do not adapt, do not cut, do not add
Three-step process：
1. Segment splitting: split into segments of about 4 seconds according to reading rhythm, mark segment_break
2. Character list/clue list: generate detailed descriptions, sync to project.json
3. Generate JSON: output the segments structured script
Each step requires user confirmation before continuing
Video Prompts contain only character dialogue, and do not include narration (narration is dubbed manually in post-production)

Output：

drafts/episode_{N}/step1_segments.md
drafts/episode_{N}/step2_character_clue_tables.md
scripts/episode_N.json(using the segments array)

Episodic animation mode (drama)

Use the Task tool to call novel-to-storyboard-script agent：

Core principles: adapt the novel into script form
Four-step process：
1. Normalize the script: structured organization
2. Shot budget: scene complexity analysis, mark segment_break
3. Character list/clue list: generate detailed descriptions, sync to project.json
4. Storyboard table: output the scenes structured script
Each step requires user confirmation before continuing
Video Prompts include dialogue, narration, and sound effects

Output：

drafts/episode_{N}/step1_normalized_script.md
drafts/episode_{N}/step2_shot_budget.md
drafts/episode_{N}/step3_character_clue_tables.md
scripts/episode_N.json(using scenes array)

Review checkpoint: After the Agent completes, display the script summary and clue list, and wait for confirmation.

Data layering explanation：

The Agent will sync the characters' and clues'complete definitionsto project.json
The script only retains characters_in_segment/scene and clues_in_segment/scene name lists
Subsequent skills read character/clue information from project.json Read character/clue information

Stage 3: Clue confirmation

After the agent completes:

Display the list of automatically identified potential clues
The user confirms which ones need to be finalized
Set the importance level (major/minor)
Update the project.json in clues field

Review checkpoint: Confirm the clue list is complete.

Stage 4: Character design

Call /generate-characters：

Generate character design images based on the description (3:4 portrait, three-view)
Save to characters/
Update project.json

Review checkpoint: Display each character and allow regeneration.

Stage 5: Clue design

Call /generate-clues：

Generate design images for the clues of importance='major' (16:9 landscape）
Save to clues/
Update project.json

Review checkpoint: Display each clue design image and allow regeneration.

Stage 6: Storyboard image generation

Call /generate-storyboard：

Mode	Process	Aspect ratio
narration	Generate storyboard images directly	9:16 portrait
drama	Two steps: multi-grid → individual scene images	16:9 landscape

Use character and clue reference images to maintain consistency
Save to storyboards/

Review checkpoint: Display each storyboard image and allow regeneration.

Stage 7: Video generation

Call /generate-video：

Mode	Aspect ratio	Default duration	Prompt content
narration	9:16 portrait	4 seconds	Dialogue only, no narration
drama	16:9 landscape	8 seconds	Dialogue + narration + sound effects

Each clip/scene is generated independently, using the storyboard image as the starting frame
Automatically use ffmpeg to concatenate into a complete video
Save to videos/ and output/
Support resumable transfer (--resume）

Review checkpoint: Preview each video and allow regeneration.

Stage 8: Final composition

Call /compose-video：

Concatenate all video clips
Apply transition effects (using segment_break markers)
Add background music (optional)
narration mode: Merge manually recorded narration
Output to output/

Project status command

Check project status at any time:

from lib.project_manager import ProjectManager
pm = ProjectManager()

# 加载项目元数据
project = pm.load_project("my_project")
print(project['status'])

# 同步并更新状态
pm.sync_project_status("my_project")

Clue management

Use clues in scenes

Add to the script clues_in_segment or clues_in_scene field:

{
  "segment_id": "E1S3",
  "characters_in_segment": ["姜月茴"],
  "clues_in_segment": ["玉佩", "老槐树"],
  "image_prompt": { ... },
  "video_prompt": { ... }
}

When generating storyboards, clue reference images will be automatically included in the API call.

Add new clue

pm.add_clue(
    "my_project",
    name="玉佩",
    clue_type="prop",  # 或 "location"
    description="翠绿色祖传玉佩...",
    importance="major"
)

Resume after interruption

The workflow can be resumed after interruption:

All progress is saved to disk
project.json records the overall project status
The script JSON records the generated resources for each segment/scene
Run again /manga-workflow to continue