whiteboard-video-workflow

Whiteboard Video Workflow

An automated workflow from an SRT subtitle file to complete whiteboard animation video clips.

Input parameters

Parameter	Required	Description
srtPath	Yes	Absolute path of the SRT subtitle file
outputDir	No	Output root directory, defaults to the directory where the SRT file is located

Workflow steps

The entire process is divided into 10 steps, which must be executed strictly in order. Steps 3, 5, and 7 use a subagent, while the remaining steps are executed directly by the main agent.

Step 0: Environment pre-check

Run this skill's scripts/check_env.py, to check all dependencies at once (Python virtual environment, RUNNINGHUB_API_KEY):

python3 <skill-dir>/scripts/check_env.py

• Success (exit code 0): capture from the output PYTHON_PATH=<路径>，record this path for use in step 7, continue
• Failure (exit code 1): from the output's ENV_RESULT=... parse each check result from the JSON,show the user a clear and easy-to-understand error explanation and repair guidance(see the troubleshooting guidance below), terminate the workflow

Note: The script automatically detects and installs fixable dependencies, and only reports issues that cannot be automatically fixed to the large model.

Troubleshooting guidance

When the environment pre-check fails, you must clearly explain the reason to the user and give specific repair steps. Depending on the failed item, provide the corresponding explanation:

RUNNINGHUB_API_KEY not configured:

Whiteboard animation video generation requires the RunningHub API to call AI models to generate images first. You need to configure the API Key in the .env file in the skill directory.

Repair method:

1. In <skill-dir>/.env add one line to the file:RUNNINGHUB_API_KEY=你的API密钥
2. If the file does not exist, create it first
3. The API Key can be obtained at RunningHub Obtain
4. After configuration is complete, just run it again

Python dependency installation failed:

Whiteboard animation depends on Python libraries such as OpenCV, NumPy, and PyAV to process video. Automatic installation was unsuccessful.

Repair method:

1. Please confirm that your Python version is 3.9 or higher
2. Then run this workflow again, and the script will try to install automatically once more

When multiple checks fail at the same time, list each failed item and its corresponding repair method one by one, so the user can solve all problems at once.

Step 1: Determine the output directory

• If the user did not specify outputDir, then use srtPath the directory where it is located as the output root directory
• Convert outputDir to an absolute path

Step 2: Create the output directory structure

Run this skill's scripts/workflow_helper.py：

python3 <skill-dir>/scripts/workflow_helper.py init-dirs "<outputDir>"

The output JSON contains storyboardDir、imageDir、videoDir three absolute paths, save them for later use.

Step 3: Parse the SRT to generate a storyboard script (subagent)

Start a subagent, with the instruction:

Use the Read tool to read the file <将本 skill 目录替换为实际绝对路径>/references/storyboard-parser.md, and execute according to the workflow steps in it.

Input parameters:

• srtPath = <将srtPath替换为实际绝对路径>
• projectRoot = <将storyboardDir替换为实际绝对路径>
• skill-dir = <将本 skill 目录替换为实际绝对路径>(used to locate the script)

After completion, return the absolute path of storyboard.json and the number of scenes.

Note: the main agent must fill in the actual path values in the instructions, do not pass variable names, the subagent cannot access the main agent's context.

Must wait for the subagent to complete and obtain the storyboard.json path before continuing.

Step 4: Parse storyboard to generate image prompts

Run this skill's scripts/workflow_helper.py：

python3 <skill-dir>/scripts/workflow_helper.py gen-prompts "<storyboardJsonPath>"

Output a JSON string array, each element is an image generation prompt with a whiteboard-style prefix, and the array index corresponds one-to-one with the order of scenes in the storyboard.

At the same time, extract each scene's duration value (in milliseconds) from storyboard.json, and record them in order as an array for later use.

Step 5: Batch generate whiteboard images (subagent)

Start a subagent, with the instruction:

Use the Read tool to read the file <将本 skill 目录替换为实际绝对路径>/references/image-generator.md, and execute according to the workflow steps in it.

Use batch mode, and pass the following JSON string array as the prompt parameter: <将步骤4输出的提示词JSON数组的实际内容粘贴于此>

Parameters:

• skill-dir = <将本 skill 目录替换为实际绝对路径>(used to locate the script)
• Output directory = <将imageDir替换为实际绝对路径>
• Aspect ratio = "16:9"

Note: the main agent must fill in the actual prompt content and path values in the instructions, do not pass variable names, the subagent cannot access the main agent's context.

Important: Return the list of paths of all generated images, and the order must be consistent with the order of the prompt array.

Must wait for the subagent to complete and obtain all image paths before continuing.

Step 6: Verify image order

Confirm that the length of the image path array returned in step 5 matches the number of scenes in the storyboard, and the order is correct (the i-th image corresponds to the i-th scene).

Step 7: Batch generate whiteboard animation video clips (subagent)

Start a subagent, with the instruction:

Call the whiteboard-animation skill, using batch mode.Skip environment pre-check(the main agent has confirmed the environment is ready), and directly run the batch generation script:

<将PYTHON_PATH替换为步骤0获取的实际值> <将whiteboard-animation skill目录替换为实际路径>/scripts/batch_generate.py \
  --images <imagePaths[0]> <imagePaths[1]> ... \
  --durations <durations[0]> <durations[1]> ... \
  --output-dir <videoDir绝对路径>

The main agent must fill in PYTHON_PATH and the actual absolute path of the skill directory in the instructions, do not pass variable names, the subagent cannot access the main agent's context.

Parameters:

• --images: list of image paths arranged in storyboard order (space-separated)
• --durations: list of durations corresponding one-to-one with the images (unit: milliseconds, directly use the duration values in the storyboard, no conversion needed)
• --output-dir：<videoDir绝对路径>

Important: After completion, collect <videoDir> all generated video file paths under the directory, sort by filename timestamp, and return the complete video path list to the main agent. The order must be consistent with the input image order.

Must wait for the subagent to complete and obtain all video paths before continuing.

Step 8: Merge video clips

Run this skill's scripts/workflow_helper.py, and merge all video clips in order into one complete video.Must use the PYTHON_PATHobtained in step 0 (PyAV dependency is in the virtual environment):

<PYTHON_PATH> <skill-dir>/scripts/workflow_helper.py merge-videos "<outputDir>" <videoPath1> <videoPath2> ...

• The first parameter is the output directory (the merged video is saved in this directory)
• The subsequent parameters are the video clip paths arranged in storyboard order

The output JSON contains mergedVideo(the absolute path of the merged video),totalSegments、sizeMB。

Step 9: Output the result

Output the final result, including the full path of the merged video and information for all clips:

Output format example:

{
  "mergedVideo": "/path/to/output/白板视频_20260329_120000.mp4",
  "videoSegments": [
    "/path/to/video/vid_20260329_120000_h264.mp4",
    "/path/to/video/vid_20260329_120010_h264.mp4"
  ],
  "totalSegments": 2,
  "sizeMB": 15.3,
  "outputDir": "/path/to/output"
}

Key constraints

• Step 0 must be executed before any work begins,check_env.py The script will automatically detect and install fixable dependencies, and only report unfixable issues to the large model
• Steps 3, 5, and 7 must be executed using a subagent, and the main agent waits for the results
• The PYTHON_PATH obtained in step 0 must be passed to the subagent in step 7 for use,and is also used for video merging in step 8(PyAV dependency is in the virtual environment), to avoid repeated environment checks
• Step 7 uses the batch mode of whiteboard-animation (batch_generate.py), where the subagent passes in all images and durations at once, the script generates them serially internally, and the subagent returns all video paths after completion
• The order of images and videos must strictly correspond to the order of scenes in the storyboard
• duration is used in milliseconds throughout the entire chain, from storyboard to batch_generate.py and then to generate_whiteboard.py, uniformly in milliseconds to avoid precision loss from floating-point conversion
• Step 8 uses PyAV (Python library) to merge video clips, re-encoding through H.264 to output a final video in a unified format

Resources

references/

• storyboard-parser.md - SRT storyboard parsing workflow instructions, read and executed by the subagent in step 3
• image-generator.md - Image generation workflow instructions, read and executed by the subagent in step 5

scripts/

• check_env.py - One-time environment pre-check, checking the Python virtual environment and API Key, and automatically installing fixable dependencies
• workflow_helper.py - Provides init-dirs(create directory structure),gen-prompts(parse storyboard to generate prompts),merge-videos(merge video clips) three subcommands
• generate-storyboard.py - Parse SRT + groups.json to generate storyboard.json
• generate-image.py - Model text-to-image, supporting single-image and batch concurrent modes
• prompt_template.py - Whiteboard-style prompt template