srt-remotion-video

SRT Remotion Video - Main Workflow Orchestration

Complete workflow for converting SRT subtitle files into Remotion videos.

Workflow Overview

┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────────────────┐   ┌──────────┐
│ 获取 SRT │ → │ 依赖预检 │ → │ 项目初始化│ → │ 生成分镜 │ → │ 并行 Creator 规划实现 │ → │ 合成视频 │
│ 文件路径 │   │ / 首次安装│   │          │   │          │   │ scene-plan + code    │   │          │
└──────────┘   └──────────┘   └──────────┘   └──────────┘   └──────────────────────┘   └──────────┘

Project Directory Structure

<skillRoot>/
├── SKILL.md
├── template/
├── references/
│   ├── storyboard-parser.md
│   └── scene-component-creator.md
└── scripts/
    ├── ensure-template-deps.js
    ├── init-project.js
    ├── generate-storyboard.js
    ├── generate-creator-scenes.js
    ├── generate-scenes-registry.js
    ├── scene-registry-utils.js
    ├── validate-project.js
    └── validate-scene-plan.js

<srtDir>/
├── your-file.srt
└── remotion-video-projects/
    └── {yyyy-mm-dd-hh-mm-ss}/

Path Contract

The main workflow and all SubAgents uniformly use the following absolute path conventions:

• skillRoot: current srt-remotion-video absolute path of the skill directory
• templateRoot: {skillRoot}/template
• referencesRoot: {skillRoot}/references
• scriptsRoot: {skillRoot}/scripts
• srtPath: absolute path of the SRT file provided by the user
• projectBaseDir: {dirname(srtPath)}/remotion-video-projects
• projectRoot: current project instance directory, in the format {projectBaseDir}/{projectName}/

Mandatory Requirements：

• The expanded absolute paths must be written into the SubAgent prompt; do not just pass variable names
• Phase protocol documents can only be read from referencesRoot Read
• Scripts can only be executed from scriptsRoot Execute
• All runtime state must be explicitly passed by the main Agent

Execution Flow

Step 0: Obtain the SRT File

1. Ask the user for the SRT file path
2. If the user provides a relative path, the main Agent must first determine on its own whether the file exists
3. If it exists, the main Agent must first resolve it to an absolute path, then use the resolved absolute path as the srtPath
4. If it does not exist, it must clearly inform the user that the path is invalid and require a correct path
5. All subsequent steps uniformly use the finally confirmed absolute path srtPath

Step 1: Dependency Pre-check and Project Initialization

Key: the template is always copied from inside the skill, and all work is performed in an independent project directory under the subtitle directory.

1.0 Dependency Pre-check (automatically installed on first use)

Execute:

node "{skillRoot}/scripts/ensure-template-deps.js" "{templateRoot}"

The script will:

1. Check {templateRoot}/package.json and {templateRoot}/package-lock.json
2. Check whether the template's key dependencies are installed
3. If not installed, then under {templateRoot} execute once npm install
4. If already installed, skip installation directly
5. If installation or validation fails, return a clear error and stop the main workflow

1.1 Default Behavior: Create a New Project

Execute:

node "{skillRoot}/scripts/init-project.js" --srt-path "{srtPath}"

The script will:

1. Ensure {dirname(srtPath)}/remotion-video-projects/ exists
2. Create a new project directory remotion-video-projects/{yyyy-mm-dd-hh-mm-ss}/
3. From {skillRoot}/template/ copy the template files; if the template dependencies are installed, also copy the installed dependencies
4. Output project information JSON

1.2 User-specified Project Path (only when the user explicitly specifies it)

If the user explicitly specifies a project path, use that path directly as projectRoot, skipping the default directory derivation.

1.3 Record Key Paths

Obtain from the script output or the user-specified path:

• projectRoot
• skillRoot
• templateRoot
• referencesRoot
• scriptsRoot
• srtPath

All subsequent steps use these absolute paths.

Step 2: Generate the Storyboard Script

This step must be executed using a SubAgent.

The main Agent is responsible for:

1. Calculate and expand the absolute paths:
   • storyboardReference = {referencesRoot}/storyboard-parser.md
   • storyboardScript = {scriptsRoot}/generate-storyboard.js
2. Launch a SubAgent
3. Write the actual absolute path values into the prompt

SubAgent prompt template:

你正在执行 srt-remotion-video 工作流的“分镜生成阶段”。

首先读取以下参考协议并严格按其步骤执行：
- {storyboardReference}

输入参数：
- skillRoot: {skillRoot}
- projectRoot: {projectRoot}
- srtPath: {srtPath}

重要：
1. 所有路径都已展开为绝对路径，不要自行猜测
2. 需要执行的脚本位于 {storyboardScript}
3. 完成后必须按参考协议中的“完成后返回”契约，返回结构化结果

The main workflow must wait for the returned result and read storyboard.json to verify the structure is correct.

Step 3: Use SubAgent to plan and implement scene components

Read storyboard.json, to obtain all scene data.

3.0 Pre-preparation: Compute grouping

const SCENES_PER_CREATOR = 5;
const sceneCount = storyboard.scenes.length;
const creatorCount = Math.ceil(sceneCount / SCENES_PER_CREATOR);

creatorId Generation rule is fixed as:

const creatorId = `creator-${String(index + 1).padStart(2, '0')}`;

Example:

• 1st creator: creator-01
• 2nd creator: creator-02
• 10th creator: creator-10

The main flow must always use this format, do not omit leading zeros, do not change to another naming method.

3.1 Plan Creator tasks

The main flow is responsible for scheduling:

1. Generate for each creator scenesDataPath = {projectRoot}/scene-plans/{creatorId}.scenes.json
2. Compute for each creator:
   • creatorId
   • planPath = {projectRoot}/scene-plans/{creatorId}.json
   • validateScript = {scriptsRoot}/validate-scene-plan.js
3. Execute for each creator:

node "{scriptsRoot}/generate-creator-scenes.js" \
  "{projectRoot}/storyboard.json" \
  "{creatorId}" \
  "{SCENES_PER_CREATOR}" \
  "{scenesDataPath}"

1. The main Agent writes the {scenesDataPath} absolute path into the SubAgent prompt
2. Start all creator SubAgents in parallel

3.2 Start all Scene Creators in parallel

SubAgent prompt template for each Creator:

你正在执行 srt-remotion-video 工作流的“场景规划与实现阶段”。

首先读取以下参考协议并严格按其步骤执行：
- {referencesRoot}/scene-component-creator.md

输入参数：
- skillRoot: {skillRoot}
- projectRoot: {projectRoot}
- creatorId: {creatorId}
- planPath: {planPath}
- scenesDataPath: {scenesDataPath}
- validateScript: {validateScript}

重要：
1. 所有路径都已展开为绝对路径，不要自行猜测
2. 当前 creator 的场景事实源只允许来自 {scenesDataPath}
3. 规划阶段只使用 {scenesDataPath}、{projectRoot}/cartoon-ui-style-guide.css、{projectRoot}/cartoon-ui-style-guide-reference.md 和 {skillRoot}/../remotion-best-practices/SKILL.md
4. 先生成 scene-plan JSON，再执行校验，校验通过后再编写场景组件
5. `beatPlan` 默认一段字幕对应一个 beat；如果相邻 segments 明显属于同一句连续表达，可以合并
6. 合并仅允许发生在相邻 segments 之间，禁止跳跃式组合
7. `beatPlan` 只声明 `segments` 和 `action`；实际时间必须从 scenesData[].segments 的 `relativeStart / relativeDuration` 推导
8. 场景主节奏必须绑定 scenesData[].segments[].relativeStart / relativeDuration
9. 默认保留宿主背景，在透明根层上围绕画面中部或中上区域组织主视觉；不要重建整屏背景
10. 组件接口固定为 React.FC<{ segments: Segment[] }> 且使用默认导出
11. 只负责产出 {projectRoot}/src/scenes/SceneXXX.tsx；若文件不存在则创建，若已存在则仅修改自己负责的场景文件
12. 不要手改 {projectRoot}/src/compositions/Main.tsx 或 generated-scenes.ts
13. 完成后必须按参考协议中的“完成后返回”契约，返回结构化结果
14. `remotion-best-practices` 与当前 skill 同级，固定入口为 {skillRoot}/../remotion-best-practices/SKILL.md

3.3 Wait for all Creators to finish

Confirm that all target scene component files have been generated.

componentResults is only used for task completion feedback, and is not the true source for final registration file assembly or total duration calculation.

Step 4: Synthesize video

4.1 Generate scene registration file

The normal video generation flow must not rewrite {projectRoot}/src/compositions/Main.tsx。

Template default output specifications:1920x1080 / 30fps。

• generated-scenes.ts of totalDurationInFrames is generated by generate-scenes-registry.js , do not modify manually
• Main.tsx in msToFrames must use useVideoConfig().fps to dynamically obtain the frame rate, cannot hardcode FPS constant

Execute:

node "{scriptsRoot}/generate-scenes-registry.js" \
  "{projectRoot}" \
  "{projectRoot}/storyboard.json"

Runtime contract is fixed:

• Each SceneXXX.tsx exposes the component through a default export
• generated-scenes.ts is responsible for saving start、duration、segments、Component
• Main.tsx is responsible for using <Component segments={scene.segments} /> to pass segment data to the scene component

4.2 Root.tsx total duration synchronization

Root.tsx is a read-only template file, and does not need to and must not be rewritten in the normal flow.

Requirements:

• Root.tsx must keep reading from generated-scenes.ts read totalDurationInFrames
• Do not repeatedly manually calculate or manually fill in the total frame count in the flow

4.3 Validate project artifact completeness

Must execute before rendering:

node "{scriptsRoot}/validate-project.js" \
  "{projectRoot}" \
  "{projectRoot}/storyboard.json"

If validation fails, the flow must stop and rendering must not continue.

4.4 Execute rendering

cd "{projectRoot}"
npx remotion render Main out/output.mp4

Step 5: Complete

Notify the user:

• Video has been generated
• Output path: {projectRoot}/out/output.mp4
• Number of scenes: N
• Video duration: X seconds

Debug mode and re-rendering

After the main video generation flow (steps 0–5) is complete, the user can request "debug mode" or "re-render".

Both of these operations assume that projectRoot already exists and the scene components have been fully generated.

Debug mode

Execute when the user says "debug mode", "preview mode" or similar expressions and requests to add audio.

TM.0 Obtain audio file

1. If the user has provided an audio file path, verify that the file exists and resolve it to the absolute path audioPath
2. If the user has not provided an audio file path, must ask the user
3. Verify that the file exists; if it does not exist, report back to the user and stop

TM.1 Add audio to timeline

1. Ensure the {projectRoot}/public/ directory exists
2. Copy the audio file as {projectRoot}/public/audio.mp3

mkdir -p "{projectRoot}/public"
cp "{audioPath}" "{projectRoot}/public/audio.mp3"

1. Modify {projectRoot}/src/compositions/Main.tsx：

   • Add to the import line Audio and staticFile：

     import { AbsoluteFill, Audio, Sequence, staticFile, useCurrentFrame } from "remotion";
     

   • In <AbsoluteFill style={{ backgroundColor: ... }}> add at the very beginning of the direct children:

     <Audio src={staticFile("audio.mp3")} />
     

2. Start Remotion Studio for the user to preview:

cd "{projectRoot}"
npx remotion studio

TM.2 Completion notification

Inform the user that Remotion Studio has started, the audio has been added to the timeline, and it can be previewed in the browser.

Re-render

Execute when the user says "re-render", "render again", or similar expressions.

RR.0 Remove audio from the timeline

Before rendering the final video, you must ensure that Main.tsx there is no audio component in:

1. Read {projectRoot}/src/compositions/Main.tsx
2. Check whether it contains the <Audio tag
3. If it exists:
   • Remove <Audio src={staticFile("audio.mp3")} /> this line
   • Remove from the import statement Audio and staticFile(if they are no longer used by other code)
4. If it does not exist, skip this step

RR.1 Execute rendering

cd "{projectRoot}"
npx remotion render Main out/output.mp4

RR.2 Completion notification

Notify the user that the video has been re-rendered, and the output path is {projectRoot}/out/output.mp4。

High-resolution / high-frame-rate rendering

Execute when the main video generation process has already been completed and the user requests generating a 4K, 60fps, or other version higher than the default configuration (1080p 30fps).

This operation assumes that projectRoot already exists and the scene components have been fully generated.

Key principle: separate design resolution from output resolution

All elements in the scene components (cards, icons, text, etc.) useabsolute pixel values, based on a 1920x1080 design.Directly changing Root.tsx's width/height to 3840x2160 will cause all elements to shrink in proportion within the frame. The correct approach is tokeep the design resolution at 1920x1080 unchanged, and use Remotion's --scale parameter to scale up the output resolution.

Common mistakes (prohibited)

Incorrect approach	Consequence
Change Root.tsx's width=3840 height=2160	All scene elements shrink to half their proportion
Use the --width 3840 --height 2160 CLI parameter	Same as above
Change only fps without changing totalDurationInFrames	The video only has content in the first half, and the second half is blank
Hardcode in Main.tsx const FPS = 30	After changing Root.tsx fps, the scene timing becomes disordered

HR.0 Confirm the user's requirements

Parse the user's requirements into specific output parameters:

User requirement	Root.tsx modification	generated-scenes.ts modification	Render command
4K / Ultra HD	Do not change	Do not change	--scale 2
60fps	fps={60}	totalDurationInFrames Convert proportionally	No scale needed
4K 60fps	fps={60}	totalDurationInFrames Convert proportionally	--scale 2

HR.1 Modify the frame rate (only when the user requests a high frame rate)

1. Modify {projectRoot}/src/Root.tsx the fps value in:

fps={60}  // 从 30 改为 60

1. Modify {projectRoot}/src/compositions/generated-scenes.ts the totalDurationInFrames：

// 帧数 = 原帧数 × (新fps / 原fps)
// 例如 30→60fps: 1572 × 2 = 3144
export const totalDurationInFrames = {原帧数 × 新fps / 原fps};

Key：{projectRoot}/src/Root.tsx in <Composition> 's width and height do not modify, must keep 1920 and 1080. Resolution upscaling is done by the --scale parameter at render time, rather than modifying the design resolution.

Main.tsx in msToFrames must use useVideoConfig().fps to dynamically get the frame rate. If you find Main.tsx has a hardcoded FPS constant, you must first fix it to useVideoConfig().fps, otherwise after the fps changes the scene timing will be completely disrupted.

HR.2 Verification Items

node "{scriptsRoot}/validate-project.js" \
  "{projectRoot}" \
  "{projectRoot}/storyboard.json"

If verification fails, you must stop and must not continue rendering.

HR.3 Execute Rendering

cd "{projectRoot}"
npx remotion render Main out/output-4k.mp4 --scale 2

• --scale 2: upscale the 1920x1080 design canvas by 2x and render as 3840x2160
• Vector elements (text, SVG) will be rendered at higher resolution, resulting in clearer image quality
• The layout is exactly the same as 1080p, with no issue of elements being scaled down
• If you don't need 4K and only need 60fps, remove --scale 2 and it's fine

HR.4 Completion Notification

Notify the user:

• Output path: {projectRoot}/out/output-4k.mp4
• Output resolution, frame rate, duration
• If fps was modified, remind the user that the fps in Root.tsx has been changed from 30 to the target value

Data Structure Reference

storyboard.json

interface Storyboard {
  totalDuration: number;
  sceneCount: number;
  scenes: {
    id: string;
    startTime: number;
    duration: number;
    segments: {
      text: string;
      relativeStart: number;
      relativeDuration: number;
    }[];
    semanticTags?: string[];
    visualHint?: string;
  }[];
}

scene-plan JSON

interface ScenePlanCard {
  sceneId: string;
  goal: string;
  layout: string;
  visualCore: string;
  surface: string;
  emphasis: string;
  screenShouldShow: string[];
  beatPlan: {
    segments: number[];
    action: string;
  }[];
}

SceneComponentResult

interface SceneComponentResult {
  sceneId: string;
  componentPath: string;
  componentName?: string;
  planPath?: string;
}

Resources

template/

• Lightweight template project, distributed together with the skill
• On first use, run dependency installation in the template directory; subsequent projects reuse the template dependencies

references/

• storyboard-parser.md: Storyboard generation phase protocol
• scene-component-creator.md: Scene planning and implementation phase protocol
• theme-template-switching.md: Theme template replacement guide

scripts/

• ensure-template-deps.js: Check template dependencies, perform first-time installation if necessary
• init-project.js: Initialize the project based on srtPath : Initialize the project based on SRT and groups.json to generate storyboard.json
• generate-storyboard.js: Generate storyboard.json based on SRT and groups.json
• generate-creator-scenes.js: Generate scenesData JSON for the specified creator based on storyboard.json
• generate-scenes-registry.js: Generate generated-scenes.ts
• scene-registry-utils.js: Shared utilities for registry and validation
• validate-project.js: Pre-render integrity verification
• validate-scene-plan.js: Verify the scene-plan JSON structure and segment bindings

Execution Checklist

Main Flow

• Get the absolute path of the SRT provided by the user
• Run ensure-template-deps.js Check template dependencies, complete first-time installation if necessary
• Run init-project.js --srt-path Create the project
• Get projectRoot、skillRoot、templateRoot、referencesRoot、scriptsRoot
• Use references/storyboard-parser.md Generate storyboard.json
• Verify storyboard.json structure is correct
• Compute Creator grouping
• Use references/scene-component-creator.md Generate scene-plan and scene components in parallel
• Run generate-scenes-registry.js
• Run validate-project.js
• Execute rendering

Debug Mode

• Get the absolute path of the audio file provided by the user
• Copy the audio to {projectRoot}/public/audio.mp3
• Modify Main.tsx Add <Audio> component
• Start Remotion Studio for user preview

Re-render

• Check Main.tsx whether <Audio> tag exists
• If it exists, remove <Audio> and related imports
• Execute rendering

High Resolution / High Frame Rate Rendering

• Parse the user's requirements into specific output resolution, frame rate, and scale values
• Confirm Main.tsx in msToFrames use useVideoConfig().fps rather than hardcoded constants
• Only when high frame rate is needed: modify Root.tsx 's fps and generated-scenes.ts 's totalDurationInFrames
• Confirm Root.tsx 's width/height keep 1920/1080 unchanged
• Run validate-project.js to validate
• Use --scale 2(at 4K) perform rendering

Notes

1. All paths must use absolute paths
2. The SubAgent prompt must pass actual path values, not just variable names
3. Template resources are located at {skillRoot}/template
4. The template is distributed in a lightweight form; on first use, template/ dependency pre-check must first be completed
5. The default project directory is located at {dirname(srtPath)}/remotion-video-projects
6. Main.tsx、Root.tsx belongs to the protected host layer
7. Scene components must actually consume segments
8. validate-project.js If it fails, rendering must not continue
9. If the user wants to change the theme template, they must first refer to references/theme-template-switching.md