ffmpeg-auto-assemble

ffmpeg auto-assemble final video · Auto-assemble with ffmpeg

storyboard.json → voiceover (edge-tts) → Ken Burns per shot / video cropping → xfade transitions → ASS subtitle kinetic text → BGM ducking mix → two-pass loudnorm (-14 LUFS) → out/video.mp4 + .srt。

Part of ecommerce-video-skills · Scripts are all in this directory scripts/：assemble.py(main program) · tts_edge.py · make_subs.py · make_bgm.py(royalty-free synthesized BGM) · make_cards.py(PIL product card/end card)

Inputs Inputs

• storyboard.json(can be generated by selling-point-storyboard ), asset paths are relative to this file
• Environment: Python 3.9+,pip install pillow numpy edge-tts, ffmpeg (with libass), CJK fonts

Quick start Quick start

pip install pillow numpy edge-tts
python scripts/assemble.py examples/aura-bottle/storyboard.json -o out/aura.mp4       # 仓库自带示例，约 30 秒渲染完
python scripts/assemble.py storyboard.json --tts none -o out/preview.mp4             # 离线预览（无配音，按字数估时）
python scripts/assemble.py storyboard.json --size 1080x1080 -o out/main-1x1.mp4      # 主图视频 1:1
python scripts/assemble.py storyboard.json --bgm music.mp3                           # 用你有版权的音乐

storyboard fields Fields

Field	Description
Top-level voice / rate / pitch	edge-tts voice, rate (e.g. +10%), pitch
Top-level bgm	"auto"(built-in synthesis, royalty-free) / music file path / null
Top-level tag	sticker in the top-left corner throughout (brand/model/"demo")
image or video (+start)	asset; video is captured from start seconds for the duration of this shot
line	voiceover text; shot duration = voiceover + 0.15s lead-in + pad
sub	subtitle (default same as line）；"" means no subtitle for this shot;**高亮**、`
title / cta	kinetic text / call-to-action button
motion	zoom_in zoom_out pan_left pan_right pan_up pan_down punch static
fit	auto(crop to fill if aspect ratio is close, otherwise blur-fill) / cover / blur
transition	xfade name for entering the next shot:fade slideleft slideup smoothleft circleopen wipeleft …
min / duration / pad	minimum duration / fixed duration / pause after sentence (seconds)

Steps Steps

1. Check that all assets exist and have suitable aspect ratios (product images ≥ 1000px); if the end card is missing, use make_cards.py to generate it: python scripts/make_cards.py --image main.png --name "商品名" --points "卖点一|卖点二|卖点三" -o assets/end-card.png
2. First run --tts none to check the visual pacing, then render for real.
3. After rendering, extract frames to check:ffmpeg -ss 3 -i out/video.mp4 -frames:v 1 check.png
4. Hand it to platform-spec-export to export versions for each platform; use cover-title-ab to make the cover.

Pipeline details How it works

• Ken Burns: frame-by-frame scale(eval=frame)+crop, less jitter than zoompan ;punch = 0.35-second fast push-in (for hooks only).
• Transitions: render an extra half transition length per shot, xfade is centered on the shot boundary,the voiceover start is not eaten by the transition。
• Mixing: voiceover is placed at each shot's start; BGM defaults to -20 dB, and is reduced by another 10 dB when there is speech (smooth ducking); compress/limit first, then two-pass loudnorm to about -14 LUFS, TP -1.5 dB.
• Subtitles: generate using the actual voiceover duration build/timing.json → make_subs.py → burn in with libass, and also export .srt。

Checklist Checklist

• final video duration, resolution, and frame rate are correct (ffprobe out/video.mp4）
• extract 3 frames: subtitles do not cover the product and do not enter the bottom UI area
• listen once: voiceover is clear, BGM does not overpower, no popping at the start
• BGM has commercial licensing (auto synthesized music can be used with confidence)
• assets (images/videos/fonts) have usage authorization

FAQ and fixes Failure fixes

Issue	Fix
ffmpeg failed	Check the end of the stderr printed by the script; common causes are a wrong path or ffmpeg missing libass/libx264
Blurry edges in the image	The source material resolution is too low; use an image ≥1080px or reduce the motion amplitude (change static）
Product gets cropped out	Set this shot to "fit": "blur"
Audio gets cut at transitions	This will not happen (voiceover is aligned to the start of each shot); if you customized duration too short, just lengthen it
Rendering is slow	The preset already uses veryfast intermediate files; for final encoding you can change -preset medium to fast
Want 1:1 / 16:9	--size 1080x1080 / --size 1920x1080, subtitle font size is automatically scaled proportionally

References and acknowledgements: the idea was inspired by MoneyPrinterTurbo(MIT) "script → voiceover → subtitles → compositing" pipeline; this implementation is an independently written pure ffmpeg version and does not copy its code.