SuperAPI AI Directory
Partner中文AI newsFavorites
Skillscode-mv-render

code-mv-render

Design & creation

View repository

Updated 10-21 06:53

About this skill

Code-rendered MV project library (two continuous deterministic music video reference projects)

This Skill does not generate single-page demos; it is a **reference library of "full-length MV rendering projects"**: it collects two open-source music video projects that share the same underlying principle —each frame is merely a function of song time（frame = renderFrame(t)), so the real-time preview in the browser and the offline-exported final cut are frame-identical, repeatedly reproducible, and precisely quality-checkable. When making a synchronized music video, choose from here the project closest to your needs to fork / adapt, rather than building a rendering pipeline from scratch.

What you are

  • Aimed at "full-length, with audio track, frame-by-frame deterministic" music videos (MV / lyric video / generative visuals), not storyboard demo pages.
  • What is provided is not template code, buttwo validated reference projects + their rendering methodology: visual rules, beat alignment, light effects and motion blur, flash safety, export quality check.
  • Boundary with video-shot-demos / swiss-shot-demos : those two are "one shot, one HTML" demo snippets; this Skill covers theentire engineering chain from audio track to final cut(analysis → timeline → scenes → preview → export).

Two reference projects

world-execute-me-lyric-mv pdoom-video
Type Lyric MV (dialogue narrative) Generative MV (word-by-word karaoke typography)
Visual technology Canvas 2D + WebGL2 (browser rendering) three.js + TypeScript (GPU scenes)
Visual language Three kinds: chatinterface / dark warm paper background with three-inkprintmaking / code(EXECUTION section) One scene module per piece: linear line art, paper form, neon screen, burning fuse…
State system Color = state（off/on/warm/error/errWarm/ash/dead） Beat-snapped scene windows + word/syllable beat hits
Repeatability renderFrame(t), PRNG-seeded, does not read the wall clock Deterministic function per frame; preview = export
Audio-visual sync data/lyric_timing.json Line-level timing + cues.js beat points; LRC auto lyric filling Demucs stem separation + CTC forced alignment × Whisper cross-validation; word-level timing
Export Headless Chrome frame by frame → ffmpeg (1080p60 / 4K60) Headless Chrome → WebSocket raw frames → ffmpeg (1080p60 / 4K60, sub-frame motion blur)
Highlight mechanisms Production-gradeflash safety(WCAG 2.3.1 self-check), error comparison thumbnail chart Block-adaptive sub-frame sampling-based motion blur, dual Chrome + single ffmpeg pipeline
Runtime environment Node.js 20+，Windows 11（Chrome/Edge） bun + Vite, Chrome + ffmpeg(libx264); analysis requires uv/Python
License Code/docs MIT; character silhouettes CC BY-NC-SA 4.0;Song lyrics not in the repository Code MIT; fonts OFL;Song lyrics not covered by MIT

Shared principles (six iron rules for making an MV)

  1. Determinism first：renderFrame(t) Depends only on time, never reads the system clock, keeps no cross-frame state, and randomness goes through a seeded PRNG. This way previews can be scrubbed, exports are reproducible, and errors can be located frame by frame.
  2. Single time base: the whole film uses "song time" (audio track start = 0). Front matter such as the opening warning page goes at negative time or under a global preroll offset, and beat points are never moved.
  3. Everything locked to the beat: lyric lines, drum hits, and per-character/syllable timings are centralized in one table; scattered timings that fall off the beat are gathered in a single cues file and fixed by ear.
  4. Visual state driven by events: colors/materials change only when "something happens" (e.g., color = state); no decorative gradients.
  5. Fonts and graphics divided by role: monospace = system, serif = narrative voice, sans-serif = interface; graphics first "reify into a worldview" (woodcut / forms / line art), then talk about typography.
  6. Safety and QC built in: large-area brightness reversals ≤3 times in any one second (WCAG 2.3.1), red flashing counted separately, and after export run audio-visual sync and single-frame jump self-checks.

Core workflow

Step 0 · Choose a track

Pick one of two by content character (the two methodologies can also be mixed):

  • Dialogue / narrative / strong concept, wanting to "draw out a worldview with code" → world-execute-me-lyric-mv route (three screens + color states + woodcut texture).
  • Abstract text / counterintuitive lines / need 3D space and light, with lyric typography as the protagonist → pdoom-video route (per-character karaoke + GPU scenes + motion blur).

Step 1 · Clear assets and licensing (critical, do first)

Neither project includes the song or lyrics, so you must supply your own:

  • world-execute-me-lyric-mv: put the original WAV and an LRC/plain text with any timeline into 音頻&歌詞/；tools/fill_lyrics.mjs only requires "the same words, in the same order", and will automatically reflow/relineate according to the in-film schedule.
  • pdoom-video: ships with a sample audio track and word-level timing data; to change songs you need to rerun analysis/(Demucs + alignment).

Confirm licensing before release: footage rendered from upstream source code contains character silhouettes → must credit CC BY-NC-SA 4.0 and must not be used commercially; prompt-only with self-drawn imagery → no legal requirement, but please note the source in the format; songs always follow the original author's copyright guidelines (personal, non-commercial, labeled as containing AI-generated content).

Step 2 · Reproduce and align

git clone target project → install dependencies per its README (npm / bun) → npm run preview or bunx vite to start live preview. When changing lyrics/beats, touch only theschedule and cues files, and leave the visual code to renderFrame(t)。

Step 3 · Preview → Export → QC

  1. Confirm section by section in the browser preview (both projects support seeking by second, frame stepping, and scene jumps).
  2. Export:npm run export:preview / export:4k（world-execute）or bun scripts/render.ts video --samples auto --shutter 0.2（pdoom）。
  3. QC: run the flash safety self-check (npm run check / tools/check_flash.py), audio-visual sync offset, and single-frame jumps;world-execute There is also a cell-by-cell regression comparison of thumbnail sheets before and after changing shared files.

Reusable key points (extracted for self-built projects)

  • Motion blur via "block-level adaptive sampling": divide the frame into 32×32 blocks; static blocks take 12 samples, regular camera moves 36, fast whips/strobe flashes 108–324, and once a block converges, skip its remaining subframes—fast objects over a static background cost almost nothing extra (pdoom's --samples auto）。
  • Preview and export share the same source: the previewer directly reuses the export frame function, eliminating "preview looks good, export looks bad".
  • Color = statepalette design: character colors (bg/panel/raised/line/text/sub/mute/me/meHot/meDim/err/errDim) switch with story events, letting viewers track the same entity across pages by color.
  • Production-grade flash budget: written into config as maxFlash / flashRange / preroll; the export script automatically generates warning pages and warning text into the README and the opening.
  • Thumbnail sheet regression: before changing the shared engine, produce a baseline thumbnail sheet; after the change, compare cell by cell—places you didn't intend to change should differ by not a single cell.

Red lines

  • Songs and lyrics are not in the repository, nor covered by the code license: the original song/lyrics must not be packaged and distributed together.
  • Character silhouettes and the frames they appear in are CC BY-NC-SA 4.0: attribution + non-commercial + share-alike under the same license.
  • The finished videos of both projects are labeled "Contains AI-generated content": this must be explicitly declared when publishing.
  • Upstream are allunofficial fan works, unrelated to the rights holders, and must not imply endorsement.
  • Don't treat platform-specific implementations (Windows paths,ffmpeg-static) as portability promises—cross-platform requires your own adaptation.

Token discipline

  • Do not read throughreference project source code (world-execute 's src/engine/、src/scenes/ and pdoom's app/src/ are both large blocks of source code).
  • For rendering methodology, prefer this Skill's references/ two condensed specs; do not read the source on the spot to reverse-engineer it.
  • When you need to look at a piece of implementation, first grep locate the function/file name, then read ≤80 lines by line-number window.

Directory structure

code-mv-render/
├── SKILL.md / README.md              ← 本文件与人类文档
├── references/
│   ├── world-execute-spec.md         ← 歌词 MV 工程规范摘录（画面三型/颜色状态/版画/剪影/确定性/闪光安全/命令）
│   └── pdoom-spec.md                 ← 生成式 MV 工程规范摘录（引擎/场景 API/运动模糊采样/时间线/渲染管线/命令）
└── assets/preview/                   ← 预览截帧（cover + pdoom plates）

Upstream sources

  • world-execute-me-lyric-mv — github.com/Nyankomintsu/world-execute-me-lyric-mv (lyric MV for Mili's "world.execute(me);", v4)
  • pdoom-video — github.com/mexicat/pdoom-video ("I'm Upping My P(doom)" generative MV)