SuperAPI AI Directory
Partner中文AI newsFavorites
Skillssrt-remotion-video-papercraft

srt-remotion-video-papercraft

Design & creation

View repository

Updated 10-16 06:51

About this skill

SRT Remotion Video

Convert an SRT into a complete video with A-roll Remotion animations as the main body and paper-art stop-motion B-roll as a supplement. The SRT timeline is always the sole authority.

Process

SRT
→ 依赖预检与项目初始化
→ Storyboard SubAgent 语义分镜并提出 A/B 类型
→ A/B 分镜校验
→ Gate 1：用户审批 A/B 方案
→ B-roll Director 生成全部图片
→ Gate 2：用户审批全部图片
→ B-roll Director 生成全部视频并准备媒体
→ Gate 3：用户审批全部视频
→ 检测可用的视频高清化 Skill，并询问是否启用
→ 可选 B-roll 高清化与 A-roll Creator 并行执行
→ 等待 A-roll 与已启用的高清化分支全部完成
→ 生成 A/B 联合注册表
→ 最终校验与渲染

When there is no B-roll, skip the B-roll Director and Gate 2, Gate 3. As long as B-roll exists, the A-roll Creator must not be started until all B-roll videos are approved.

Path Contract

  • skillRoot: absolute path of the current Skill
  • templateRoot: {skillRoot}/template
  • referencesRoot: {skillRoot}/references
  • scriptsRoot: {skillRoot}/scripts
  • srtPath: absolute path of the user-provided SRT
  • projectRoot: default {dirname(srtPath)}/remotion-video-projects/{yyyy-mm-dd-hh-mm-ss}

All Agent prompts must pass expanded absolute paths. Phase rules can only be read from the corresponding reference, and runtime state must be persisted to disk.

0. Input and Initialization

After validating srtPath , execute:

node "{scriptsRoot}/ensure-template-deps.js" "{templateRoot}"
node "{scriptsRoot}/init-project.js" --srt-path "{srtPath}"

Templates are always copied to a separate project; old projects do not automatically sync with new templates.

1. Storyboard SubAgent

Start a SubAgent and pass in:

  • {referencesRoot}/storyboard-parser.md
  • skillRoot
  • projectRoot
  • srtPath
  • {scriptsRoot}/generate-storyboard.js
  • {scriptsRoot}/validate-roll-plan.js

The SubAgent must strictly execute the Storyboard reference and generate groups.json and storyboard.json, and make the A/B storyboard validation pass. The main Agent verifies the structured return result.

2. Gate 1: Approve the A/B plan

The main Agent displays each scene's ID, time range, subtitle summary,rollType、rollReason、visualHint, as well as the total B-roll duration and suggested proportion.

When the user changes the type:

  1. Update groups.json
  2. Rerun generate-storyboard.js
  3. Rerun validate-roll-plan.js
  4. Redisplay Gate 1

After the user approves, the current storyboard.json becomes a frozen source of truth. The SRT timeline must not be modified based on media generation results.

3. B-roll Director and Gate 2, Gate 3

Executed only when the Storyboard contains B-roll. The main Agent resolves the installed papercraft-stop-motion-explainer entry absolute path, and starts a B-roll Director SubAgent, passing in:

  • {referencesRoot}/broll-director.md
  • papercraftSkillPath
  • projectRoot
  • {projectRoot}/storyboard.json
  • {projectRoot}/broll-production.json
  • {scriptsRoot}/prepare-broll-asset.js
  • {scriptsRoot}/validate-broll-assets.js

The main Agent holds the user session and all Gates; the B-roll Director is responsible for distilling a unified paper-art art direction and a Shared Style Anchor that can be reused verbatim from the entire batch of B-roll, then generating prompts for each scene according to the image-series consistency rules, invoking the generation Skill, and completing asset persistence and manifest updates. This internal art-direction distillation does not add Gates, style preview images, or reference image processes. When the user requests changes, prefer sending a follow-up to the same Director; if it is unavailable, the new Director must recover from the manifest.

Strict order:

  1. The Director generates all images and returns.
  2. The main Agent displays all images (with the local absolute path of each image) and executes Gate 2.
  3. The user modifies scene by scene until all images are approved.
  4. Only then can the Director generate all videos, run the media preparation script, and return.
  5. The main Agent displays all videos (with the local absolute path of each video) and executes Gate 3.
  6. The user modifies scene by scene until all videos are approved.
  7. Return the approval results to the Director, who updates the manifest and runs validate-broll-assets.js; the main Agent continues only after confirming the structured validation result.

When an image is modified, invalidate only that scene's video and return to Gate 2. On failure, explain the reason and feasible suggestions, and wait for the user to decide; automatic retries, degradation, reverting to A-roll, or adding a static registration mode are prohibited.

4. Optional B-roll Upscaling and A-roll Creator

After Gate 3 passes, the main Agent checks whether the currently available Skills in the environment include a Skill that explicitly supports video enhancement or video super-resolution, without hardcoding a specific Skill name.

  • No available Skill: inform the user and directly start the A-roll Creator.
  • One found: explain the discovered capability and ask the user whether to enable it.
  • Multiple found: list the candidates and let the user choose.
  • User skips: directly start the A-roll Creator.
  • User agrees: create an independent {projectRoot}/broll-upscale.json, start an Upscale SubAgent, and execute it in parallel with all A-roll Creators.

Enhancement is a pluggable post-processing module and must not modify the original B-roll assets,broll-production.json , or the Gate 3 approval status. For the detailed protocol, see {referencesRoot}/broll-upscaler.md. The main Agent must pass the absolute entry path of the selected Skill as upscaleSkillPath to the Upscale SubAgent; the same SubAgent is responsible for the entire batch of B-roll and no separate Agent is started for each scene.

After the user agrees to enhancement, the main Agent completes startup preparation according to the manifest contract of broll-upscaler.md :

  1. Confirm that all Storyboard B-roll are broll-production.json in video-approved。
  2. Create decision: accepted, batch status: pending of broll-upscale.json, recording the selected Skill name and absolute entry path.
  3. For each B-roll, create a per-scene pending record and fixed asset path, selecting the enhanced version by default.
  4. Change the batch status to running, then start the Upscale SubAgent and the A-roll Creator simultaneously.

The specific per-scene fields, asset paths, media information, and source file fingerprints use only broll-upscaler.md as the source of truth, andSKILL.md does not redefine them. When the user skips or no available Skill exists locally, the main Agent still records the corresponding decision, so that the recovery process does not ask again. Old projects missing this file are treated as having enhancement not enabled.

4.1 A-roll Creator

Read the frozen Storyboard and count only the number of A-roll:

aRollCount = scenes.filter(rollType !== "b-roll").length
creatorCount = ceil(aRollCount / 5)

For each creator-XX execute:

node "{scriptsRoot}/generate-creator-scenes.js" \
  "{projectRoot}/storyboard.json" \
  "{creatorId}" \
  "5" \
  "{projectRoot}/scene-plans/{creatorId}.scenes.json"

Start Creator SubAgents in parallel, passing in:

  • {referencesRoot}/scene-component-creator.md
  • skillRoot
  • projectRoot
  • creatorId
  • scenesDataPath
  • planPath
  • {scriptsRoot}/validate-scene-plan.js

All rules for the Creator use the reference as the source of truth. The Creator only generates the assigned A-roll SceneXXX.tsx, and must not modify the host, registry, or B-roll.

4.2 Parallel Convergence

After all A-roll Creators complete:

  1. If enhancement is not enabled, proceed directly to registration.
  2. If enhancement is still pending / running, wait for the Upscale SubAgent to complete.
  3. If any scene or batch is failed, report the failed scenes, reasons, and available handling options to the user; automatically using the original video, automatically retrying, or proceeding to registration is prohibited.
  4. Only when all scenes that selected the enhanced version are completed, or the failed scenes have been explicitly changed by the user to the original video, and {scriptsRoot}/validate-broll-upscale.js passes, may registration proceed.

After a failure, the user may explicitly choose to retry the failed scenes, fall back to the original video per scene, or abandon all enhancement; the main Agent must first write the decision into the enhancement manifest according to the contract of broll-upscaler.md , and then continue. There is no Gate 4: after all enhancement tasks and technical validation are complete, the enhanced assets are used automatically.

5. Registration, Validation, and Rendering

After all A-roll complete, execute:

node "{scriptsRoot}/generate-scenes-registry.js" \
  "{projectRoot}" \
  "{projectRoot}/storyboard.json"

node "{scriptsRoot}/validate-project.js" \
  "{projectRoot}" \
  "{projectRoot}/storyboard.json"

Rendering is prohibited when validation fails. After it passes, execute:

cd "{projectRoot}"
npx remotion render Main out/output.mp4

Notify the user of the output path, number of scenes, number of A/B scenes, and total duration.

Recovery Rules

If projectRoot exists, read it first:

  • storyboard.json
  • broll-production.json(if present)
  • scene-plans/
  • src/scenes/

Continue based on completed Gates and asset status; do not repeat approved production. Any rollType modification must regenerate the Storyboard from groups.json , re-validate, and recalculate Creator grouping; unaffected approved B-roll assets may be retained.

If broll-upscale.json：

  • accepted + running/pending: resume or wait for the same batch of upscaling tasks.
  • accepted + failed: wait for user instructions; do not automatically roll back.
  • accepted + completed: after validation, use the upscaled assets.
  • skipped / unavailable: use the original B-roll; do not ask again.
  • When the original clip.mp4 changes, the corresponding upscaling result is invalidated due to source fingerprint mismatch.
  • If the recorded upscaling Skill entry is unavailable, rediscover a Skill with the same name or capability, and do not replace the implementation on your own before user confirmation.

Debugging, re-rendering, and high resolution

  • All B-roll must be muted at the Remotion compositing layer;BrollScene.tsx 's <OffthreadVideo> must set muted, and the B-roll asset's own audio track must not be written into the final cut.
  • Copy the debug audio to public/audio.mp3, and add a global Main.tsx at the <Audio>host layer; B-roll remains muted.
  • During the final re-render, remove only the global debug audio; do not remove the B-roll's muted setting.
  • Output specifications come from src/video-settings.json; design coordinates are always 1920x1080.
  • 4K uses the existing output profile or --scale 2; do not rewrite the Creator layout.
  • After changing fps, the registry must be regenerated so that totalDurationInFrames matches the fps.

Asset responsibilities

  • references/storyboard-parser.md: semantic storyboard and A/B decisions
  • references/broll-director.md: B-roll image/video production and Gate status
  • references/broll-upscaler.md: optional B-roll upscaling post-processing and recovery protocol
  • references/scene-component-creator.md: A-roll planning and implementation
  • scripts/validate-roll-plan.js: A/B data and 4-8 second hard validation
  • scripts/prepare-broll-asset.js: media probing and tail frame extraction
  • scripts/prepare-upscaled-broll-asset.js: normalize upscaling results, media probing, source fingerprints, and upscaled tail frame extraction
  • scripts/validate-broll-assets.js: B-roll asset and approval status validation
  • scripts/validate-broll-upscale.js: optional upscaling status, assets, media info, and source fingerprint validation
  • scripts/broll-upscale-utils.js: independent upscaling manifest reading and final render asset resolution
  • scripts/generate-creator-scenes.js: filter and assign A-roll
  • scripts/generate-scenes-registry.js: A/B joint registration
  • scripts/validate-project.js: final integrity validation

Core invariants

  1. The SRT timeline, total duration, and Scene ID do not change with media generation results.
  2. The content duration of a single B-roll scene must be 4-8 seconds; a proportion of about 15% is recommended, with no hard ratio limit.
  3. Keep hard cuts between A/B scenes; do not add visual transitions.
  4. B-roll video plays at original speed; if shorter than the slot, freeze the real tail frame, and if longer than the slot, cut it off at the start of the next scene.
  5. B-roll assets may contain audio tracks, but they must be muted during final compositing; the final cut's audio may only come from global audio tracks explicitly mounted at the host layer.
  6. Final rendering only accepts standard B-roll video that has passed Gate 3.
  7. Legacy projects missing rollType are handled as A-roll-only compatible.
  8. When upscaling is not enabled, the original B-roll production, registration, and rendering behavior remains unchanged.
  9. Upscaling must not rewrite the Storyboard; ultimately, use the real duration and real last frame of the upscaled file to execute the existing cropping and frame interpolation rules.