SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillstalking-head-remotion

talking-head-remotion

Design & creation

View repository

Updated 10-15 23:59

About this skill

Talking Head Remotion

A Studio-style Remotion talking-head video template: the scaffold generates a runnable demo project in one step, paired with a cross-project shared asset library (fonts, SFX, BGM, reusable motion components).

Input requirements

  • Final talking-head audio or talking-head video (required): it is the single source of truth for the entire timeline; subtitles, element entrances, and SFX are all derived from it. When the user only provides content/topic without audio, first help the user write a script and generate TTS, then follow the same process; do not lay out timing out of thin air.
  • Script/verbatim transcript (strongly recommended): not used to set timing, but to proofread the transcribed subtitles and mark keywords and chapter divisions.
  • Screen recordings, screenshots, SFX, and BGM briefs are optional and can be added later at any time.

New project: run the scaffold first

python3 ./skills/talking-head-remotion/scripts/scaffold_talking_head_remotion_project.py \
  --project-dir "/path/to/new-project" \
  --title "视频标题" \
  --script "/path/to/script.md" \
  --talking-head "/path/to/koubo.mp4" \
  --screen-recording "/path/to/recording1.mp4" \
  --screenshot "/path/to/screenshot1.png"

Only --project-dir is required; assets can be added later. The generated project is ready to use out of the box npm install && npm run still && npm run render:preview, and fonts and common SFX are already seeded.

Project directory conventions

Asset Location
Talking-head portrait video public/media/talking-head/talking-head.mp4
Screen recording public/media/screen-recordings/
Image/screenshot assets public/assets/screenshots/
Voice audio public/assets/audio/voice.m4a
Sound effects SFX public/assets/audio/
BGM public/assets/music/
Fonts public/assets/fonts/
Motion components copied from the asset library src/library/
Script/subtitle/planning documents work/

The Remotion entry point is src/index.ts, the composition is registered in src/Root.tsx , the demo data is in src/demoData.ts, and design variables and non-blocking font registration are in src/theme.ts。

Generated asset archiving

If TTS or subtitles were first generated in the notes vault, they must be archived back to the corresponding video engineering project before delivery. The current 《两个提示词让 AI 少返工》 project directory is:

<VIDEO_WORKSPACE>/your-talking-head-project

Use these fixed locations when archiving:

  • Final voiceover:public/assets/audio/voice.m4a; also keep public/assets/audio/voice.mp3 for ASR, editing software, or quick preview.
  • TTS segments and concatenation list:work/audio/generated/。
  • TTS-specific script:work/tts/。
  • Original verbatim transcript or source script:work/source/。
  • Main subtitles:work/captions/captions.srt、work/captions/captions.vtt。
  • Raw ASR and alignment data:work/captions/captions.raw.srt、work/captions/captions.raw.vtt、work/captions/asr-result.json、work/captions/captions_aligned.json。

Shared asset library

Assets reused across projects are placed in this skill's assets/library/; for the list and accumulation rules, see assets/library/LIBRARY.md。

  • Fonts and SFX are automatically seeded into the project by the scaffold.
  • Motion components (such as the big-text slam TitleSlam.tsx) are copied into the project on demand src/library/ and then imported for use.Use only when appropriate: library components and components already built in the project are only cost-saving tools, not inventory that must be consumed—when the specs/storyboard require a new visual, make a new one; do not substitute ready-made components as a makeshift replacement (criterion: if this ready-made component did not exist, would you create the same visual).
  • For new motion/sound effects verified as useful in a project, copy them back to the library after delivery and register a line in LIBRARY.md (what they are suitable for and what they are not), so the asset library keeps growing.

Workflow

  1. Align audio: when there is only talking-head video, first extract the voice ffmpeg -y -i public/media/talking-head/talking-head.mp4 -vn -c:a aac public/assets/audio/voice.m4a. Check the opening silence/offset; if necessary, first generate the aligned voice and PIP assets, then transcribe the subtitles; do not compensate piecemeal in the subtitles or animation code.
  2. Subtitle-driven timeline: the corrected subtitles are written to work/captions/captions_aligned.json, then synchronized to src/demoData.ts. Chapter transitions, keyword appearances, screen-recording entries, and CTAs are all aligned to the start of the corresponding subtitle.
  3. Implementing the visuals: write the Remotion code according to references/visual-guide.md 's design variables and fixed layers.First look for work/制作规格.md(a mandatory deliverable of video-script for animation-pipeline projects): if present, reproduce it table by table—implement components per the spec sections, configure each shot's content per the "per-shot state table/content table", and draw concrete graphics per the "drawing recipes",do not improvise, do not simplify, do not merge states; only fill in details not written in the spec yourself, and write them back into the spec once done. If this file is absent and the scene contains simulated UI/concrete graphics/components that evolve with the story, stop first and produce the spec (per video-script's references/production-spec.md), and do not write code directly from the textual descriptions in the storyboard table—the historical lesson is "simulated UI turned into empty boxes, panels statically spoiling the plot, a parrot drawn as an ink blob" (J-space first version).
  4. Validation：npm run typecheck + npm run still; then first run a low-resolution proof (default npm run render:preview) to check the entire timeline, subtitles, audio, and scene pacing. Self-check by extracting frames from the proof (one frame per chapter + one per high-density shot): check item by item against work/制作规格.md 's acceptance checklist; two mandatory checks—the element count in the 0.3s opening frame of a multi-element scene must differ from that in a mid-section frame; no conceptual element may appear in any frame before its story point (to prevent spoilers).Before rendering, first run an audio-visual word-alignment audit: the cues for conceptual elements must be looked up by subtitle text (cueByText style); sentence-order indexing and fixed-delay fallbacks are forbidden (that is the root cause of "the parrot appearing before the word 'parrot'", except for decorative skeletons); check element by element that appearAt falls within the [字幕开始-0.3s, 字幕开始+1s] window; after rendering, extract "1s before / 0.5s after" dual frames for each conceptual element's first-appearance line, verifying respectively that it is absent/present. If there is a constant offset across the whole film, check the audio source (opening silence, whether the aligned audio and the final audio are the same file), and do not compensate piecemeal in the animation code. Only after the user explicitly requests "generate the final version/official export" or the low-resolution proof has been confirmed, run npm run render。
  5. Add sound last: lock SFX and BGM only after the visuals and animation beats are stable (see below).
  6. Accumulate: useful motion effects/sound effects flow back into the asset library; record lessons that can prevent future mistakes into the project work/lessons/LESSONS.md。

Render debugging rhythm

  • By default, produce a low-resolution proof first; do not immediately run a full 1920x1080 official render of the whole film. For the low-resolution proof use renders/preview-low.mp4, usually --scale=0.5 --crf=28 --concurrency=2 is enough to find subtitle misalignment, empty frames, volume issues, and scene pacing issues.
  • Before the official render, do at least three types of checks:npm run typecheck, keyframe stills, and a low-resolution full-film proof. When the user has not explicitly said "final version/official export", stop at the proof and do not proactively spend time running full-res.
  • For long renders, redirect the output to work/render.log, and only tail the tail of the log; do not flush thousands of lines of per-frame progress into the conversation context.
  • Do not use vague CLI probing commands, for example running directly in the project remotion render --help. This command will be treated as starting the default render in some versions/contexts. When checking parameters, prioritize looking at package.json existing scripts, project README, or official documentation; when you need to try a command, first make sure it will not start a render.
  • After the user interrupts, first check background processes:pgrep -fl "remotion|chrome-headless|Google Chrome for Testing|Chromium". Interrupt an accidentally started render, wait for or clean up Remotion/Chrome headless, and do not leave behind processes eating CPU/GPU.

Non-negotiable rules

  • Subtitles, PIP lip sync, animation beats, and SFX must all be checked against the same final aligned audio source.
  • PPT-style full-page appearance is forbidden: when a scene cuts in, it must not lay out all the content at once. Every visible element (a line of title, a list item, a number, an icon, a screenshot) must have its own entrance animation and entrance moment, and the entrance order is driven by the subtitle order—only when the voiceover reaches a piece of content does that element pop out. A scene change only swaps the stage; elements still enter one by one.
  • Main-screen text should be sparse rather than abundant (≤5 text elements per screen, and it can have no text): only put keywords, phrases, numbers, evidence screenshots, or short CTAs; full sentences go only in subtitles, and putting the verbatim script on screen is forbidden.
  • Richness comes from non-text layers, not from adding words: each screen must have at least one non-text visual subject (diagram/simulated UI/screen recording/data visualization/graphic animation); at any moment at least one layer of continuous motion must be moving; the same layout must not continue for more than 2 scenes; when talking about something specific that can be screenshotted/recorded, a real or simulated interface must be provided. Motion effects must cover the four categories of entrance/continuous/emphasis/illustrative, and cannot be only entrance (see visual-guide's "画面丰富度硬规则" and "动效词汇表" for details).
  • The talking-head portrait is a circular PIP in the bottom right (showing head and shoulders); when there is no video, use a neutral placeholder.
  • The default aspect ratio is 16:9 landscape (1920x1080); the template's fixed layers (top progress bar, bottom-right PIP, subtitle safe area, perspective grid) are all designed for landscape, and making vertical (9:16) requires separately adjusting these layers—it is not ready out of the box. Default 1920x1080 full-frame Studio theme: warm white canvas + top and bottom dual-layer mirrored perspective grid background; do not wrap the content in a white rounded card, shrink the canvas, or use a webpage preview container.PremiumGridBackground must actually call PerspectiveGrid。
  • Animations use only useCurrentFrame() + interpolate() + Sequence frame-driven; do not use CSS animation/transition or browser timers (HyperFrames' GSAP timeline cannot be copied directly).
  • Assets go in public/ and are referenced with staticFile() ; fonts use local @font-face non-blocking registration or a stable system font stack by default. Do not use @remotion/fonts/loadFont() by default to block rendering of the whole video, because it depends on delayRender(), and occasional font loading stalls will make long videos fail with timeouts at arbitrary frames. Use it only when blocking font consistency is truly needed, and first verify with a low-resolution proof.
  • The initial template's middle stage is intentionally left empty (only the background, top bar, PIP frame, and subtitle style examples); during production, fill the scene element by element according to the subtitles, while the spatial system, fixed layers, and background structure remain unchanged.

Sound effects and BGM

Place them after the visuals are complete. SFX and BGM are not decoration; they prevent the pacing from being too flat: every sound must be able to answer "what pacing problem did it solve," and do not add sound effects to every subtitle.

  • First fill in work/audio/audio_cue_sheet.md(sound effect, start, duration, visual event, pacing purpose, volume), then in the composition use Sequence to wrap <Audio> placement, starting with conservative volume.
  • Vocals volume={1} always have the highest priority; BGM preview listening 0.06-0.10, always below the vocals.
  • Use instrumental music for BGM (110-128 BPM is common for tech/explainer), write in the brief work/music/bgm_brief.md, after generation save to public/assets/music/ and add SOURCES.md; for the final mix, render first and then use work/music/mix_bgm.sh for ducking.