seedance-storyboard

Seedance 2.0 Storyboard Creation and Video Generation

The full process from idea to final video: guide the storyboard → draw reference images → submit video → get results.

Step 0: Determine the execution method (MCP or script)

First check whether MCP is available:

Check xskill-ai MCP service status (read mcps/user-xskill-ai/STATUS.md）
If MCP is available → use submit_task / get_task and other MCP tools
If MCP is unavailable or returns an error → switch toscript mode

Prerequisites for script mode:

Confirm the environment variable XSKILL_API_KEY is set (Shell execution echo $XSKILL_API_KEY | head -c 10）

If not set, prompt the user:

export XSKILL_API_KEY=sk-your-api-key
获取 API Key: https://www.xskill.ai/#/v2/api-keys

Confirm requests is installed (pip install requests）

Script path: In this Skill directory, scripts/seedance_api.py, located by:

# 通过 Glob 工具搜索
glob: .cursor/skills/seedance-storyboard/scripts/seedance_api.py

In subsequent steps, each API call provides both the MCP method and the script method, and one of them is chosen for execution based on the judgment in Step 0.

Step 1: Understand the user's idea

Collect the following information (proactively ask for anything missing):

Story core: summarize in one sentence what is to be filmed
Duration: 4-15 seconds
Aspect ratio：16:9 / 9:16 / 1:1 / 21:9 / 4:3 / 3:4
Visual style: realistic/animation/ink wash/sci-fi/cyberpunk, etc.
Material status: whether there are ready-made images/videos/audio, or whether AI generation is needed
Feature mode: whether first-last frame control (first_last_frames) is needed; otherwise default to omni mode (omni_reference)

Step 2: Deep dive (5 dimensions)

For each dimension, guide the user to add details:

Content - Who is the protagonist? Doing what? Where?
Visuals - lighting, color tone, texture, atmosphere
Camera - push/pull/pan/track/follow/orbit/crane
Action - the subject's specific actions and rhythm
Sound - music style, sound effects, dialogue

Step 3: Build the storyboard structure

Split shots along the timeline, using the following formula:

【风格】_____风格，_____秒，_____比例，_____氛围

0-X秒：[镜头运动] + [画面内容] + [动作描述]
X-Y秒：[镜头运动] + [画面内容] + [动作描述]
...

【声音】_____配乐 + _____音效 + _____对白
【参考】@image_file_1 _____，@video_file_1 _____

For detailed templates and examples, see reference.md。

Step 4: Generate reference images (if needed)

If the user has no ready-made materials, use Seedream 4.5 to generate character images, scene images, first/last frames, etc.

Text-to-image

Call the submit_task tool:

model_id: fal-ai/bytedance/seedream/v4.5/text-to-image
parameters:
- prompt: detailed image description (English works better)
- image_size: choose according to the video aspect ratio
- num_images: number needed (1-6)

python .cursor/skills/seedance-storyboard/scripts/seedance_api.py submit \
  --model "fal-ai/bytedance/seedream/v4.5/text-to-image" \
  --params '{"prompt":"An astronaut in a white spacesuit...","image_size":"landscape_16_9","num_images":1}'

Image editing (modify based on existing images)

Call the submit_task tool:

model_id: fal-ai/bytedance/seedream/v4.5/edit
parameters:
- prompt: editing instructions (reference images as Figure 1/2/3)
- image_urls: array of input image URLs
- image_size: output size

python .cursor/skills/seedance-storyboard/scripts/seedance_api.py submit \
  --model "fal-ai/bytedance/seedream/v4.5/edit" \
  --params '{"prompt":"Change the background to a forest","image_urls":["https://..."],"image_size":"landscape_16_9"}'

Poll for image results

Images complete in about 1-2 minutes.

Call the get_task tool to check status:

First check after 30 seconds
Then check every 30 seconds
Status is completed Extract the image URL.

Single query:

python .cursor/skills/seedance-storyboard/scripts/seedance_api.py query \
  --task-id "TASK_ID_HERE"

Automatic polling (recommended for images, interval 10s, timeout 180s):

python .cursor/skills/seedance-storyboard/scripts/seedance_api.py poll \
  --task-id "TASK_ID_HERE" --interval 10 --timeout 180

image_size reference table

Video ratio	Recommended image_size	Description
16:9	landscape_16_9	Landscape
9:16	portrait_16_9	Portrait
4:3	landscape_4_3	Landscape
3:4	portrait_4_3	Portrait
1:1	square_hd	Square
21:9	landscape_16_9	Approximate ultrawide

Step 5: Generate the professional prompt

Integrate the storyboard structure and reference images into the final prompt:

Use @image_file_1、@image_file_2 etc. to reference images in the image_files array
Use @video_file_1 etc. to reference videos in the video_files array
Use @audio_file_1 etc. to reference audio in the audio_files array
Also compatible with the old @图片1、@视频1、@音频1 syntax

Reference syntax examples:

@image_file_1 作为角色形象参考，参考 @video_file_1 的运镜方式，配合 @audio_file_1 的配乐

Important: image_files The Nth URL in the array corresponds to @image_file_N，video_files and audio_files are numbered independently.

Step 6: Submit the video task

Process the asset URLs:

Images generated by Seedream: already have URLs, use directly
Network images provided by the user: use directly
Local images provided by the user: upload first to get a URL (see the upload method below)

Upload local images

Call the upload_image tool: image_url or image_data

# 上传网络图片
python .cursor/skills/seedance-storyboard/scripts/seedance_api.py upload \
  --image-url "https://example.com/image.png"

# 上传本地图片
python .cursor/skills/seedance-storyboard/scripts/seedance_api.py upload \
  --image-path "/path/to/local/image.png"

Submit a Seedance 2.0 task (omni mode omni_reference)

Call the submit_task tool:

model_id: st-ai/super-seed2
parameters:
- prompt: the complete prompt generated in Step 5
- functionMode: omni_reference(default, can be omitted)
- image_files: array of reference image URLs (up to 9, in order corresponding to @image_file_1/2/3...)
- video_files: array of reference video URLs (up to 3, total duration ≤ 15s)
- audio_files: array of reference audio URLs (up to 3)
- ratio: aspect ratio (16:9 / 9:16 / 1:1 / 21:9 / 4:3 / 3:4）
- duration: duration as an integer (4 - 15）
- model: seedance_2.0_fast(default fast) or seedance_2.0(standard quality)

python .cursor/skills/seedance-storyboard/scripts/seedance_api.py submit \
  --model "st-ai/super-seed2" \
  --params '{
    "prompt": "电影级写实科幻风格，15秒，16:9...",
    "functionMode": "omni_reference",
    "image_files": ["https://img1.png", "https://img2.png"],
    "ratio": "16:9",
    "duration": 15,
    "model": "seedance_2.0_fast"
  }'

Submit a Seedance 2.0 task (first and last frame mode first_last_frames)

Call the submit_task tool:

model_id: st-ai/super-seed2
parameters:
- prompt: video description prompt
- functionMode: first_last_frames
- filePaths: array of image URLs (0 = text-to-video, 1 = first-frame image-to-video, 2 = first-and-last-frame video)
- ratio: aspect ratio
- duration: duration as an integer
- model: seedance_2.0_fast or seedance_2.0

python .cursor/skills/seedance-storyboard/scripts/seedance_api.py submit \
  --model "st-ai/super-seed2" \
  --params '{
    "prompt": "镜头从首帧缓缓过渡到尾帧，画面流畅自然",
    "functionMode": "first_last_frames",
    "filePaths": ["https://first-frame.png", "https://last-frame.png"],
    "ratio": "16:9",
    "duration": 5,
    "model": "seedance_2.0_fast"
  }'

Step 7: Poll and wait for the video result

Video generation takes about 10 minutes.

Polling strategy:

After submitting, tell the user "The video is being generated, estimated 10 minutes"
For the first time, 60 secondslater, call get_task to query
After that, every 90 secondsquery once
Report the status to the user after each query

Recommended: use poll for automatic polling (runs in the background, interval 30s, timeout 600s):

python .cursor/skills/seedance-storyboard/scripts/seedance_api.py poll \
  --task-id "TASK_ID_HERE" --interval 30 --timeout 600

The script prints polling progress to stderr and outputs the JSON result to stdout when complete.

Manual query in multiple steps:

python .cursor/skills/seedance-storyboard/scripts/seedance_api.py query \
  --task-id "TASK_ID_HERE"

Status description:

pending → "Queued..."
processing → "Generating..."
completed → Extract the video URL and show it to the user
failed → Inform the user of the failure reason and suggest adjusting the prompt and retrying

Complete Workflow Example

User says: "Help me make a video of an astronaut walking on Mars"

When MCP Is Available

1. 收集信息 → 15秒，16:9，电影级科幻风格，无现成素材

2. 用 Seedream 4.5 生成宇航员图 + 火星场景图
   submit_task("fal-ai/bytedance/seedream/v4.5/text-to-image", {...})
   → 轮询 get_task → 获得图片 URL

3. 编写提示词 → 提交视频任务
   submit_task("st-ai/super-seed2", {...})

4. 轮询 get_task，~10分钟后获得视频 URL

When MCP Is Unavailable (Script Mode)

1. 收集信息 → 15秒，16:9，电影级科幻风格

2. 生成参考图：
   python scripts/seedance_api.py submit \
     --model "fal-ai/bytedance/seedream/v4.5/text-to-image" \
     --params '{"prompt":"An astronaut in white spacesuit on Mars...","image_size":"landscape_16_9"}'
   → 拿到 task_id

3. 轮询图片结果：
   python scripts/seedance_api.py poll --task-id "xxx" --interval 10 --timeout 180
   → 拿到图片 URL

4. 提交视频任务：
   python scripts/seedance_api.py submit \
     --model "st-ai/super-seed2" \
     --params '{"prompt":"...分镜提示词...","functionMode":"omni_reference","image_files":["图片URL"],"ratio":"16:9","duration":15,"model":"seedance_2.0_fast"}'
   → 拿到 task_id

5. 轮询视频结果：
   python scripts/seedance_api.py poll --task-id "xxx" --interval 30 --timeout 600
   → 拿到视频 URL

Model Parameter Quick Reference

Seedream 4.5 Text-to-Image

Parameter	Type	Required	Description
prompt	string	Yes	Image description
image_size	string	No	auto_2K / auto_4K / square_hd / portrait_4_3 / portrait_16_9 / landscape_4_3 / landscape_16_9
num_images	int	No	1-6, default 1

Seedream 4.5 Image Editing

Parameter	Type	Required	Description
prompt	string	Yes	Editing instruction, reference with Figure 1/2/3
image_urls	array	Yes	List of input image URLs
image_size	string	No	Same as above
num_images	int	No	1-6, default 1

Seedance 2.0 Video (All-Around Mode omni_reference)

Parameter	Type	Required	Description
prompt	string	Yes	Storyboard prompt, reference with @image_file_N/@video_file_N/@audio_file_N
functionMode	string	No	`omni_reference`(default)
image_files	array	No	Array of reference image URLs (up to 9)
video_files	array	No	Array of reference video URLs (up to 3, total duration ≤ 15s)
audio_files	array	No	Array of reference audio URLs (up to 3)
ratio	string	No	21:9 / 16:9 / 4:3 / 1:1 / 3:4 / 9:16
duration	integer	No	Integer 4-15, default 5
model	string	No	seedance_2.0_fast (default) / seedance_2.0

Seedance 2.0 Video (First/Last Frame Mode first_last_frames)

Parameter	Type	Required	Description
prompt	string	Yes	Video description prompt
functionMode	string	Yes	`first_last_frames`
filePaths	array	No	Array of image URLs (0 = text-to-video, 1 = first frame, 2 = first and last frames)
ratio	string	No	21:9 / 16:9 / 4:3 / 1:1 / 3:4 / 9:16
duration	integer	No	Integer 4-15, default 5
model	string	No	seedance_2.0_fast (default) / seedance_2.0

Tool Quick Reference

MCP Tools

Operation	Tool	Key Parameter
Submit task	submit_task	model_id, parameters
Query result	get_task	task_id
Upload image	upload_image	image_url or image_data
Query balance	get_balance	None

Script Commands (When MCP Is Unavailable)

Operation	Command	Description
Submit task	`python scripts/seedance_api.py submit --model MODEL --params '{...}'`	Returns task_id
Single query	`python scripts/seedance_api.py query --task-id ID`	Returns current status
Automatic polling	`python scripts/seedance_api.py poll --task-id ID --interval N --timeout N`	Blocks until complete
Check balance	`python scripts/seedance_api.py balance`	Return account balance
Upload image	`python scripts/seedance_api.py upload --image-url URL` or `--image-path PATH`	Return image URL

Script path notes: In the above commands, scripts/seedance_api.py is a path relative to the .cursor/skills/seedance-storyboard/ directory. When actually executing, use the full path .cursor/skills/seedance-storyboard/scripts/seedance_api.py, or first cd to the skill directory.

Seedance 2.0 limitations

Uploading realistic human face material is not supported
Maximum 12 files: images ≤ 9 + videos ≤ 3 + audio ≤ 3
Total video/audio reference duration ≤ 15 seconds
Including video references will consume more credits

More resources

For detailed storyboard templates, complete examples, and a shot vocabulary list, see reference.md。

About this skill

Step 0: Determine the execution method (MCP or script)

Step 1: Understand the user's idea

Step 2: Deep dive (5 dimensions)

Step 3: Build the storyboard structure

Step 4: Generate reference images (if needed)

Text-to-image

Image editing (modify based on existing images)

Poll for image results

image_size reference table

Step 5: Generate the professional prompt

Step 6: Submit the video task

Upload local images

Submit a Seedance 2.0 task (omni mode omni_reference)

Submit a Seedance 2.0 task (first and last frame mode first_last_frames)

Step 7: Poll and wait for the video result

Complete Workflow Example

When MCP Is Available

When MCP Is Unavailable (Script Mode)

Model Parameter Quick Reference

Seedream 4.5 Text-to-Image

Seedream 4.5 Image Editing

Seedance 2.0 Video (All-Around Mode omni_reference)

Seedance 2.0 Video (First/Last Frame Mode first_last_frames)

Tool Quick Reference

MCP Tools

Script Commands (When MCP Is Unavailable)

Seedance 2.0 limitations

More resources