Input a long technical article (or an expandable topic) → Output a 1080p chapter-based explainer MP4:
- Layout: Each chapter has three layout options, mixed to create a sense of rhythm—
illustration(default): Left column title + key point cards + terminology tags, right column AI illustrations (full-chapter Ken Burns slow push in and pull out)grid: The right column is replaced with a card grid, each block grows in one by one with the narration (used for step/comparison/checklist chapters, strong informational feel)statement: No right column, full-screen large-text judgment sentences (used for short, punchy chapters with ≤3 sentences, as breathing points)terminal: macOS-style terminal window in the right column, commands enter line by line with a typewriter effect (used for chapters explaining CLI; HTML renders directly, commands will never have typos)- Bottom sentence-by-sentence subtitles + chapter progress bar + large gray chapter number in the upper right are common to all layouts
- Among 8-10 chapters, it is recommended that 2-4 chapters use grid/statement, if the whole video is all large images it will drag down the rhythm, and completion rate will be poor
- Visuals: cozy-handdrawn warm hand-drawn style — cream paper background, coral red accents, low-saturation pastels, hand-drawn offset shadow cards; illustrations use inline-diagram's cozy-handdrawn style (edit mode + LQ IP reference image), page color scheme is in the same family as the illustrations, unified overall
- Pacing: 6-10 chapters, each chapter 30-60 seconds, whole video 5-8 minutes
Pipeline overview (5 steps)
# 0. 写分镜 (必读 references/storyboard-guide.md)
# 读文章 → <project_dir>/storyboard.json: 每章 kicker/headline/subhead/card/tags/
# narration(句子数组)/illustration.brief。分镜质量决定成片质量。
# 1. 插图 (需要 MULERUN_API_KEY; 三个脚本跨 skill 复用)
python3 scripts/make_manifest.py <project_dir> # → diagrams.json
python3 ../linyuebanzi-inline-diagram/scripts/inject_style.py \
--style cozy-handdrawn --manifest <project_dir>/diagrams.json # → diagrams-styled.json
python3 ../linyuebanzi-image-gen/scripts/generate.py \
--manifest <project_dir>/diagrams-styled.json \
--output-dir <project_dir>/images --parallel # → images/ch-NN.png
# 1b. 插图去底 (需要 Pillow) — 抠掉纯色背景, 透明 PNG 浮在页面上, 换背景直接兼容
python3 scripts/strip_background.py <project_dir>/images # 原图备份到 images/raw/
# 2. 逐句 TTS (需要 MINIMAX_API_KEY; 无 key 用 --provider say 预览)
python3 scripts/tts_pipeline.py <project_dir> # → audio/ + timeline.json
# 3. 合成 index.html (全数据驱动, 不需要手写场景)
python3 scripts/scaffold.py <project_dir> # → index.html
# 4. 出片
bash scripts/build.sh <project_dir> # → renders/<name>.mp4 + 蒙太奇
Note: the paths of inject_style.py / generate.py are resolved according to the actual skill installation location (they respectively belong to
linyuebanzi-inline-diagram and linyuebanzi-image-gen)。
Environment: Node.js ≥22 + ffmpeg (npx hyperframes doctor self-check); MULERUN_API_KEY(illustrations) +
MINIMAX_API_KEY(voiceover).
Theme (storyboard's theme field)
| theme | Visuals | Illustration strategy |
|---|---|---|
warm(default) |
Cream paper background + coral red + pastels | cozy-handdrawn + Remove background(transparent floating) |
midnight |
Deep blue-black background + amber gold + large Songti headings (AI tech style) | executive-tech + Do not remove background(dark frame) |
For the same storyboard, just change the theme field to change the skin; layout/animation/timeline all inherit. The difference is only in step 1:
- warm:
inject_style.py --style cozy-handdrawn→ generate image →strip_background.py - midnight:
inject_style.py --style executive-tech→ generate image → Skip background removal(black-line hand drawings are invisible on a dark background, dark design images come with their own background, directly framed)
To add a new theme: copy a body[data-theme="..."] variable block in the template + choose an inline-diagram style, a ten-minute job.
Voiceover timbre
Default male voice male-qn-jingying(LQ is a male blogger), speech rate 1.05. Change it in the voice block of storyboard.json.
Use your own voice: Minimax supports voice cloning, clone once and it can be used for life:
# 录一段 10s~5min 的干净人声 (安静环境、无 BGM、自然语速, 3 分钟左右最佳)
python3 scripts/clone_voice.py my-voice.m4a --voice-id LQvoice2026 \
--preview-text "大家好，我是林月半子。"
# 完成后把 voice_id 填进 storyboard.json 的 voice.voice_id
Note: Minimax charges a one-time fee when a cloned voice is synthesized for the first time; cloned voices can only be used for content authorized by the person themselves.
Pacing
Timeline constants are in tts_pipeline.py Top: HEAD_PAD 0.7 / GAP 0.18 / TAIL_PAD 0.75.
The right-side illustration does a Ken Burns slow push/pull throughout the chapter (alternating direction on odd/even chapters), so the frame never sits still.
If it still feels too slow: first raise voice.speed(1.05→1.15), then cut the number of narration sentences; don't touch the template animation durations.
Why sentence-by-sentence TTS
Write narration as anarray of sentences, with each sentence TTS'd separately. The resulting sentence-level timeline drives three things at once:
- Subtitles synced sentence by sentence(like the reference video; a single block of subtitles looks stiff)
- Key-point card item anchoring: the
at: Nin card.items means that item enters when sentence N starts - Change one sentence, rerun only that sentence: use
--skip-existing, delete the corresponding mp3 and rerun
Iteration conventions
| What changed | What to rerun |
|---|---|
| A line of narration | Delete the corresponding audio/ch-NN-sNN.mp3 → tts_pipeline.py --skip-existing → scaffold.py → build.sh |
| A chapter illustration | Edit the storyboard brief → make_manifest.py → inject+generate (keep only the item to rerun) → directly build.sh |
| Card/title copy | scaffold.py → build.sh(no need to rerun TTS) |
| Add/remove chapters/sentences | Rerun the whole pipeline (the timeline changed) |
storyboard.json structure (full spec in references/storyboard-guide.md)
{
"topic": "j-lens",
"title": "AI 没说出口的想法，第一次被直接读到了",
"voice": { "provider": "minimax", "voice_id": "female-chengshu", "speed": 1.0 },
"chapters": [
{
"id": 1,
"kicker": "开场 / AI 黑箱",
"headline": ["AI 没说出口的想法", "第一次被直接读到了"],
"subhead": "Anthropic 的 J-lens，正在把模型内部的沉默概念变成可读信号",
"card": {
"title": "先记住这件事",
"items": [
{ "label": "以前", "text": "只能看 AI 说了什么", "at": 2 },
{ "label": "现在", "text": "开始能看它没说什么", "at": 4, "bar": 0.7 }
]
},
"tags": ["J-space", "J-lens", "silent thoughts"],
"narration": [
"你大概会把自己跟AI的聊天记录翻出来，看看它到底说了什么。",
"但你看到的，永远只是它说出口的那部分。"
],
"illustration": {
"id": "ch-01",
"brief": "{插图内容提示词, 不含风格前缀, 写法见 storyboard-guide}"
}
}
]
}
Artifact structure
<project_dir>/
├── storyboard.json # 分镜 (人工可改的唯一事实源)
├── diagrams.json # 插图 manifest (make_manifest 产物)
├── diagrams-styled.json # 注入 cozy-handdrawn 后
├── images/ch-NN.png # 每章插图
├── audio/
│ ├── ch-NN-sNN.mp3 # 每句配音
│ └── timeline.json # 章节+句级时间轴
├── index.html # scaffold 产物 (勿手改, 改 storyboard 重新 scaffold)
├── renders/<name>.mp4 # 成片
└── preview/montage.png # 每章中点蒙太奇 (渲染后必看)
QC checklist
build.shBuilt-in lint + validate; lint must have 0 errors- After renderingmust check
preview/montage.png: whether any chapter has an empty illustration/text overflow/ugly title line breaks - Go through the illustrations manually: whether the LQ figure is distorted, whether the Chinese annotations are garbled (if garbled, change the text in the brief to bilingual or rerun)
- Sample two chapters and listen to the voiceover: whether the sentence breaks are natural, whether numbers/English are read correctly (if wrong, change the narration wording, e.g. write "J-lens" as "J透镜")
Pitfalls from real use
| Symptom | Countermeasure |
|---|---|
| The illustration drew style words as on-image text (in practice "wobble" appeared in the image) | Don't put English adjective-style descriptions in the brief; if found, change the brief and rerun that image |
| The LQ figure's clothing drifts across multiple images (gray T/purple T/black T) | Write "LQ figure wears a gray T-shirt" uniformly in the brief; 8-10 consecutive appearances in one video are more noticeable than article illustrations |
| Calling Minimax sentence by sentence easily hits the RPM rate limit | tts_pipeline.py already has built-in 1002 rate-limit 21s backoff; this is normal, just wait for it to finish on its own |
<audio> No id means muted during rendering |
scaffold.py already auto-generates ids; don't delete them when manually editing index.html |
| Background removal also cuts out near-white panels in the illustration | Flood fill only removes color blocks connected to the edges; if the illustration has a large panel ≈ background color, it will be eaten. In the brief, explicitly give panels a pastel base color (blue/green/purple/orange), or for individual images --tolerance 12 rerun |
| The local proxy (127.0.0.1:8899) self-signed certificate causes Minimax TLS failure | The script bypasses the system proxy by default; if you need to go through a proxy export MINIMAX_USE_PROXY=1 |
Known limits
- Key-point card items ≤4, text ≤18 characters; exceeding this crowds it (the template doesn't crop, rely on storyboard discipline)
- headline ≤11 Chinese characters per line, two lines preferred
- Each chapter's narration 4-9 sentences, each sentence ≤32 characters (one subtitle line)
- Illustration aspect ratio is fixed at 4:3, object-fit contain; other ratios can also be used but will leave blank space
- The template has no silent mode (this is a commentary video; being silent makes no sense)
Resources
assets/template/index.html— Data-driven HyperFrames template (placeholders are filled in by scaffold.py)scripts/make_manifest.py— storyboard → illustration manifestscripts/tts_pipeline.py— sentence-by-sentence Minimax TTS + timelinescripts/scaffold.py— compose index.htmlscripts/build.sh— lint + render + montagereferences/storyboard-guide.md— storyboard breakdown specification (required reading for step 0)- Cross-skill dependencies:
linyuebanzi-inline-diagram(style injection) +linyuebanzi-image-gen(image generation)