SuperAPI AI Directory
Partner中文AI newsFavorites
Skillsip-talking-head-lecture

ip-talking-head-lecture

Office & productivity

View repository

Updated 10-17 15:13

About this skill

ip-talking-head-lecture: IP digital human talking-head animated courseware video production

The tool is this repository (below <仓库>), the CLI is <仓库>/bin/iph, and all parameters are in <仓库>/config.json. Each episode's content is in dirs.projects(default <仓库>/episodes/<日期 标题>/), and intermediate artifacts and the finished video are in dirs.build(default <仓库>/build/<日期 标题>/）。

Workflow in one sentence: write a script.json(one per scene narration + heading + kind + data）→ iph all → get the mp4.

Consult the details as needed in references/：

File What it covers
references/script-format.md script.json All fields and the six layout data structures (read this if you want to write a script)
references/voices.md Differences between the three TTS channels Xiaomi MiMo / Volcano / edge, how to choose a voice, and a quick error lookup
references/avatar.md Talking-head avatar: how to switch between the two built-in sets (animal cartoon / human cartoon), how to add a new avatar, and how to prepare three-frame mouth-shape assets (including how to write prompts for AI-generated mouth-shape frames)
references/publish.md Publishing copy and subtitles: how to write thepublish section, YouTube chapters' hard 10-second rule, how to align Chinese and English subtitles, and conventions for each platform
references/delivery-qa.md Finished-video acceptance checklist (machine + visual) and common pitfalls

Visual specs (factory settings; change config.json to change the video)

背景   浅色纸感（径向白→薄荷绿渐变）+ 缓慢漂浮的色块 + 极淡点阵
主色   品牌绿 #2f9e44（强调下划线、序号、高亮字、数字人光环）
字体   HarmonyOS Sans SC（标题）/ MiSans（正文）—— 都是本机已装字体
页眉   左上胶囊小标签 → 大标题 → 副标题；右上 "02 / 06" 场景进度
内容区 横屏 (104,296) 起 1316×592；竖屏 (72,330) 起 936×1030
数字人 横屏：圆形 340px，圆心 (1690,752)；竖屏：280px，圆心 (880,1520)
       形象可选：haibao=动物卡通（海老豹）｜human=人物卡通（眼镜青年）
字幕   底部居中深色圆角胶囊，普通白字 + **正在读的字用品牌色加粗**
品牌层 左下角「名称 · slogan」水印 + 右上角「✦ 博主名」角标（brand.corner）+ 顶部 5px 进度条
片尾卡 build 自动追加 outro 场景：标识图/手写博主名 + 品牌色划线 + slogan + 黄色便签 CTA（brand.endCard）

Digital humanmustbe pinned to the lower right corner and must not overlap the subtitle bar; the subtitle barmustuse a dark background (pure white text on a light background will blur). Change config.json 's layout.<画幅> to adjust the position.

Hard rules

  1. One source of truth:script.json is the only handwritten source. Narration, layout, titles, and data are all inside it.timeline.json、字幕.srt、旁白稿.md are all generated artifacts; do not edit them by hand—they will be overwritten on the next build. To adjust duration, adjust the narration length;never manually fill in frame counts or seconds。
  2. Duration is calculated, not hardcoded. Each scene's duration = leadPadding(0.85s) + 该段配音实际时长 + tailPadding(0.7s), with a lower bound of minSceneSeconds(3.2s). So changing one narration line automatically re-lays out the entire video, and audio, visuals, and subtitles are naturally aligned.
  3. Define narration first, then data。 Narration is "words to be spoken aloud" (colloquial, short sentences, no punctuation hierarchy);heading is "big on-screen text" (≤12 characters);data is short words for specific key points. Do not copy-paste among the three, otherwise the visuals are just repeating the subtitles.
  4. Self-check the voice channel first. If you changed tts , run iph voice: it echoes the engine/endpoint/model/voice and actually synthesizes one sentence, which is easier than getting Unknown voice halfway through production. The three engines each have theirownvoice fields (Volcano tts.voice / Xiaomi tts.mi.voice / edge tts.edge.voice); substituting one for another is the most common configuration pitfall.
  5. The digital human is "talking right up close", not "reading a script". The mouth opening and closing rhythm is driven by remotion/src/components/TalkingAvatar.tsx drives the pseudorandom syllable function. One image can work, falling back to slight compression, a halo and waveform bars, butthree mouth frames (closed, slightly open, open) look noticeably more lifelike. Two built-in avatar sets are available: cartoon animals (haibao) and cartoon people (human);iph avatars lists them, andiph preset <名> or --preset=<名> switches them. See references/avatar.md。
  6. Deliverables: finished video (landscape and portrait), subtitles and 旁白稿.md + 发布文案.md。 Subtitles and publishing copy are byproducts of iph build and need no separate work: produce Chinese and English SRT and VTT versions, with bilingual YouTube copy. In the user-facing response,paste the primary title and platform descriptions directlyinstead of merely saying the task is complete.
  7. YouTube chapters require a first entry at 00:00, at least three entries and at least ten seconds per entry. Failing any condition invalidates the entire chapter set. Chapters are therefore merged automatically from the storyboard (lib/publish.mjs 's groupChapters），do not hand-write timestamps. They become stale after a single narration edit. Use the {{chapters}} placeholder for every timeline in platform copy.
  8. Render without network access. Use installed local Chinese fonts and local assets. Remotion initially tries to download chrome-headless-shell; if blocked, lib/render.mjs automatically uses Chromium already installed by Playwright.
  9. Time reference:timeline.json each scene's cues / words times are relative to the start of that scene, in seconds, rather than global time (toSrt() adds scene.startonly during export). If consuming components are mounted at the top level (useCurrentFrame() receives global frames), theymust add scene.startthemselves; see IpLecture.tsx 's toGlobal(). When changing this area,inspect both aspect ratios and at least one frame from a scene after the first. In the first scene, start = 0masks every offset error.
  10. English subtitle entries reuse the Chinese subtitle timeline one for one, so publish.en.cues must have exactly as many entries as the Chinese version (iph srt <期> can count them).If counts differ, build skips English subtitles and warnsrather than forcing alignment; a mismatched timeline is worse than none.

Command

I=<仓库>/bin/iph
$I new "<标题>"                    # 建期：<episodes>/<日期> <标题>/script.json（从模板复制）
$I list                            # 列出所有期
$I voice                           # 配音自检：回显档位 + 真合成一句
$I build <期>                      # 配音 + 生成时间轴 / 字幕（中英 × SRT/VTT）/ 旁白稿 / 发布文案
$I publish <期>                    # 同 build —— 只改了 publish 段时用这个，配音走缓存秒级完成
$I still <期> [秒]                 # 出静帧看版式 → <build>/<期>/work/out/still-*.png
$I render <期>                     # 渲染成片 → <build>/<期>/outputs/final-<画幅>.mp4
$I all <期>                        # build + render 一条龙
$I srt <期>                        # 只看字幕条时间与文本（核对断句，也可用来数英文条数）
$I kinds                           # 列出可用课件版式
$I avatars                         # 列出可用口播形象（含三帧齐不齐 + 当前默认）
$I preset <形象名>                  # 切换默认口播形象（写进 config.json）
$I open <期>                       # 打开期目录与后台目录

Common switches:--ratio=16:9,9:16(aspect ratios; generating just one is allowed) |--preset=human(override the avatar for this episode) |--force-tts(ignore narration cache and regenerate) |--scene=2(render one scene for a quick layout check) |--still=2,12,25(produce multiple still frames at once).

<期> may be a directory name or any substring of a title, for example iph all 全流程。

Standard workflow

I=<仓库>/bin/iph

# 1. 建期 + 写脚本
$I new "什么是大模型"
#    → 编辑 episodes/<日期> 什么是大模型/script.json
#      scenes[].narration 写"要念的话"，heading 写"屏上大字"，kind 选版式，data 填要点

# 2. 配音自检（换了音色才需要）
$I voice

# 3. 出片（横竖两版一起出）
$I all "什么是大模型"
#    → build/<日期> 什么是大模型/outputs/final-16x9.mp4
#    → build/<日期> 什么是大模型/outputs/final-9x16.mp4

Six courseware layouts

kind determines the layout;data holds layout-specific content (see the full field list in references/script-format.md）。

kind Where to use it data
cover Cover/opening: large title, underline and tags eyebrow tags[]
idea One core concept and two to four points center icon points[{title,text}]
steps A two- to four-step process with numbered badges items[{title,text,icon}]
compare Two-column comparison with a circular VS badge left/right{title,items[]}
numbers One to three animated numbers items[{value,unit,label,prefix}] note
recap Two or three summary cards and a closing CTA items[{title,text}] cta
outro Closing brand card (no narration;brand.endCard.enabled makes build append it automatically; usually no manual entry is needed) name slogan cta logo(injected from config.brand)

icon Optional:bulb chart chip layers chat rocket target book(outline icons; see remotion/src/components/Icons.tsx）。

Outline for a typical 60–80-second video：cover → idea → steps → compare → numbers → recap. Six sections, each 6–10 seconds, with approximately 250–320 Chinese characters of narration in total.

Add a layout: in remotion/src/scenes/index.tsx write a component → register it in the file's final SCENES → in references/script-format.md add the data structure. Landscape uses side-by-side columns; portrait always stacks vertically (layout.portrait determines this).

Supporting landscape and portrait

The same script.json produces two aspect ratios:timeline.json generates one version per aspect ratio (layout.<画幅> uses different avatar positions and subtitle heights), overwriting the Remotion project before rendering. Layout components use layout.portrait to determine orientation: multiple columns in landscape, vertical stacking and one smaller font size in portrait.Test both orientations when adding a layout（iph still <期> 12 --ratio=9:16）。

Publishing copy and subtitles (iph build generates them automatically)

script.json Add a publish section; build then generates copy and subtitles together without separate work:

Artifact Contents
发布文案.md Primary title and Bilingual Chinese and English YouTube copy(title, description, tags and chapter timeline), plus introductions for WeChat Channels, Xiaohongshu, Douyin, Bilibili and WeChat Official Accounts
字幕.srt · 字幕.vtt Chinese subtitles (VTT is YouTube's preferred format)
字幕.en.srt · 字幕.en.vtt English subtitles (requires publish.en.cues）

Three key points:

  • YouTube chapters are calculated automaticallyrather than hand-written: first entry at 0:00, at least three entries,and at least ten seconds per chapter. A 60-second video with six shots often has only eight or nine seconds per shot. Listing chapters by shot causes YouTube to discard the entire set , so lib/publish.mjs 's groupChapters() automatically merges shots shorter than ten seconds into the current chapter. The chapter title comes from its first shot's heading; set youtube.chapterTitles / chapterTitlesEn。
  • Chinese and English subtitles share one timeline：publish.en.cues and the Chinese subtitles mustcorrespond one for one（iph srt <期> can count entries). If counts differ, build skips English subtitles and prints a warning; misaligned timing is worse than no subtitles.
  • The {{chapters}} placeholder in platform copy is replaced with ascene-leveltimeline, without merging, that always follows the final duration. Hard-coded timestamps such as 00:08 become stale after edits. Use placeholders for Douyin and Bilibili copy.

See the full field list and platform writing guidance in references/publish.md。

Branding and avatars

  • The brand name, slogan, primary color and watermark position are in config.json 's brand; an episode may use script.json 's brand to override them.

  • Creator-name badge in the top-right corner：brand.corner.enabled(true by default) and text(leave empty to use brand.name). One change applies throughout the video; the closing card does not show the corner badge.

  • Closing brand card：brand.endCard.enabled(true by default) makes build automatically append an unvoiced outro scene (seconds controls card duration, andcta the note text). Put the creator's logo in brand.logo(PNG/JPG path; a file with the same name in the episode directory's assets/ takes priority). Without an image, use the creator's name in large text, a brand-color underline and a slogan. Disable it with"endCard": { "enabled": false } or set the episode's script.json to "brand": { "endCard": { "enabled": false } }。

  • Two built-in talking-avatar sets，config.json 's avatar.presets registers them, andavatar.preset selects the current default:

    Preset name Avatar Asset directory
    haibao Cartoon animal: Hai Lao Bao assets/avatars/haibao/
    human Cartoon person: young man with glasses assets/avatars/human/ 
    $I avatars                      # 看列表（三帧齐不齐 + 当前默认打 *）
$I preset human                 # 切默认形象（全局）
$I all <期> --preset=human       # 只给这一期换，不动全局

    Priority:--preset= > script.json 's avatar.preset > config.json 's avatar.preset > legacy fallback path. Avatar information is written into timeline.json during build; rebuild after switching avatars to update the video（still/render with --preset= automatically builds first).

  • Each avatar directory contains three frames with the same basename:image.*(closed), mouth-mid.*(slightly open) and mouth-open.*(open), with any extension. Place episode-specific avatars in the episode directory's assets/; they take priority over preset directories.

  • Add an avatar by creating a directory with three frames → register an entry in config.json 's avatar.presets (label / dir / focus / scale）→ iph avatars to confirm. See complete steps and mouth-frame generation prompts in references/avatar.md。

Common problems

Symptom Cause and remedy
Unknown voice: xxx Voice fields from the three engines were mixed. Xiaomi has only nine fixed voices (mimo_default/冰糖/茉莉/苏打/白桦/Mia/Chloe/Milo/Dean）
45000010 Invalid X-Api-Key The Volcengine API domain is wrong (a key beginning withark- works only with the plan domain); this is not a quota issue
A whole subtitle sentence highlights at once, without word-by-word tracking Expected: the Xiaomi route provides only short-sentence timestamps. For word-level highlighting, change tts.engine to volc
spawnSync ffprobe EBUSY This machine's Nodesynchronoussubprocess calls are unavailable (the repository now uses asynchronous lib/tts/run.mjsthroughout). Do not use execFileSync
Recursive directory deletion reports SAFE_DELETE_BULK_GUARD_ERROR The local safe-delete tool blocks bulk deletion. Do not use fs.rmSync(dir,{recursive:true}); overwrite named files instead
Chromium-related rendering error Specify it manually:REMOTION_BROWSER=<chrome-headless-shell 路径> iph render <期>
Slow rendering or memory pressure Reduce config.json 's video.concurrency from 4 to 2
Avatar overlaps subtitles Adjust layout.<画幅>.avatar.y to move it higher, or adjust layout.<画幅>.subtitle.bottom to place subtitles lower
presets 里没有形象「xxx」 The name is misspelled or unregistered.iph avatars lists available options
Video still uses the old avatar after switching Avatar information is written into at build time; the destination is timeline.json . Run iph build <期> --preset=xxx(or simply iph all）
iph avatars shows a cross for the background image The avatar directory lacks image.*(or its basename differs). See references/avatar.md for the directory structure
Publishing-copy Markdown file was not generated script.json lacks a publish section (subtitles are still generated)
Warning: publish.en.cues 有 N 条，但中文字幕是 M 条 English subtitle counts differ and were skipped. Use iph srt <期> to count Chinese entries and fill English entries one for one
The publishing copy prints {{chapters}} unchanged Misspelled placeholder: two ASCII braces, case-sensitive
Warning that there are not enough shots for three chapters of at least ten seconds The video is under 30 seconds, so YouTube chapters cannot be used; omit timestamps from the description
Chinese characters appear as squares A Chinese font is missing. Install HarmonyOS Sans SC, MiSans or Source Han Sans