SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsseedance-video-creator

seedance-video-creator

Design & creation

View repository

Updated 10-16 01:25

About this skill

Seedance 2.0 storyboard video creation tool

Transform creative ideas into professional storyboard prompts, first generate reference images via the Jimeng API, then generate videos based on images + prompts, and automatically download them.

Prerequisites

Before using this skill, ensure the following services are available:

  1. jimeng-free-api-all service is running(default port 8000) 
    # Docker 部署
docker run -it -d --init --name jimeng-free-api-all \
  -p 8000:8000 -e TZ=Asia/Shanghai \
  wwwzhouhui569/jimeng-free-api-all:latest

  1. Get SessionID：

  2. Environment variables(optional, can also be specified during interaction):

    • JIMENG_API_URL: API base address (default http://127.0.0.1:8000），Note: fill in only the base address, do not include the path part. If using the duckcloud service, it is https://jimeng.duckcloud.fun
    • JIMENG_SESSION_ID: Jimeng SessionID

Core capabilities

Seedance 2.0 features

Important constraints：Seedance 2.0 At least one reference image must be provided, plain-text video generation is not supported. When the user has not provided an image, a first-frame reference image must first be generated via the text-to-image API.

Multimodal input(up to 12 files):

  • Images: ≤9 (first frame/last frame/character/scene reference)
  • Videos: ≤3, total duration ≤15s (camera movement/action/effects reference)
  • Audio: ≤3, total duration ≤15s (soundtrack/sound effects/dialogue reference)
  • Text: natural language description

Video parameters：

  • Model:seedance-2.0-fast(recommended, fast version) / jimeng-video-seedance-2.0-fast(equivalent alias) / jimeng-video-seedance-2.0(Pro version) / seedance-2.0 / seedance-2.0-pro(Pro equivalent alias)
  • Ratio:1:1 / 4:3 / 3:4 / 16:9 / 9:16 / 3:2 / 2:3 / 21:9
  • Resolution:480p / 720p(default) / 1080p
  • Duration:4 - 15 seconds

Interaction method: use @1、@2(or @图1、@image1) to specify the material usage

Complete workflow (three stages)

Core process: storyboard prompt → text-to-image generate first frame → image + prompt generate video

用户输入创意 → 第一阶段：生成分镜提示词
             → 第二阶段：调用文生图 API 生成首帧参考图 → 下载图片到本地
             → 第三阶段：用参考图 + 分镜提示词调用 Seedance 2.0 → 下载视频

Stage 1: Storyboard prompt generation

Step 1: Understand the user's idea

Ask the user in a friendly way:

  1. What story do you want to tell?(summarize the core content in one sentence)
  2. Video duration?(4-15 seconds, default 4 seconds)
  3. Do you have reference materials?(image/video path)
  4. Do you need to directly generate a video?(Yes → enter stages 2 and 3 / No → output only the prompt)

Step 2: Dig deeper into details

Based on the user's initial answers, gradually guide them to refine the following dimensions:

1. Content narrative

  • The story's setup, development, turn, and conclusion
  • Key plot points and characters
  • Dialogue/voiceover content

2. Visual style

  • Overall style: realistic/animation/ink wash/sci-fi/retro/cinematic
  • Color tone and atmosphere: bright/dim/warm/cool/black and white
  • Aspect ratio: vertical (9:16)/horizontal (16:9)/standard (4:3)

3. Camera language

  • Shot size changes: long shot → full shot → medium shot → close-up → extreme close-up
  • Camera movement: push/pull/pan/track/follow/orbit/crane/Hitchcock zoom
  • Transition method: hard cut/dissolve/match cut/effects transition

4. Action and rhythm

  • Subject action: walking/running/fighting/dancing/facial expression changes
  • Action rhythm: relaxed/urgent/with tension and release

5. Sound design

  • Music style: passionate/warm/suspenseful/cheerful/epic
  • Sound effect requirements: ambient sound/action sound effects/special sound effects

Step 3: Build the storyboard structure

Break the story down into a sequence of shots on a timeline.

Example structure (4-second video)：

0-1秒：开场镜头，建立场景
1-3秒：发展，核心动作/情感
3-4秒：结尾，画面定格

Step 4: Generate two types of prompts

Based on the collected information, generate separately:

A. First-frame image prompt (for text-to-image)

Describe the static image of the video's first frame, as the first-frame reference image for Seedance 2.0.

First-frame prompt requirements：

  • Describe a static moment of the image (excluding dynamic descriptions such as action or camera movement)
  • Include: scene, character/subject appearance, lighting, color tone, composition
  • The aspect ratio should match the video aspect ratio
  • Concise and refined, 100-200 characters

Example：

日落时分的海边沙滩，一位穿着白色飘逸长裙的年轻女孩站在海边，面朝大海，夕阳逆光照射，金色光晕笼罩全身，长发和裙摆被海风轻轻吹动，远处海平线上的落日将天空染成橘红色，脚边浅浅的海浪泡沫

B. Video storyboard prompt (for Seedance video generation)

Complete storyboard prompt, use @1 to reference the first-frame reference image.

Prompt structure template：

【整体描述】风格 + 时长 + 画面比例 + 整体氛围

@1 作为画面首帧参考

【分镜描述】
0-X秒：[镜头运动]，[画面内容]，[主体动作]，[光影/特效]
X-Y秒：[镜头运动]，[画面内容]，[主体动作]，[光影/特效]
...

【声音说明】配乐风格/音效/对白

Prompt writing principles：

  1. Clear timeline, mark the second range for each segment
  2. Clear camera language (push/pull/pan/track/follow/orbit, etc.)
  3. Specific action descriptions, avoid vague wording
  4. Must use @1 to reference the first-frame image(The first-frame image is automatically generated in the second stage)
  5. If the user provides multiple images, use @1、@2 to reference them separately
  6. Keep the total length within 500 characters

Step 5: Optimize and confirm

  • Check whether platform rules are violated (realistic face restrictions)
  • After confirming the user is satisfied, proceed to the second stage

Second stage: Generate the first-frame reference image via text-to-image

Purpose: Seedance 2.0 must have a reference image. When the user does not provide an image, first generate the first-frame image through the text-to-image API.

If the user has already provided an image: Skip this stage and proceed directly to the third stage.

Step 1: Call the text-to-image API

curl -s --max-time 120 -X POST "${API_URL}/v1/images/generations" \
  -H "Authorization: Bearer ${SESSION_ID}" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"jimeng-4.5\",
    \"prompt\": \"${首帧图片提示词}\",
    \"ratio\": \"${RATIO}\",
    \"resolution\": \"2k\"
  }"

Parameter description：

Parameter Description Recommended value
model Image generation model jimeng-4.5(stable and high quality)
prompt First-frame image description The first-frame prompt generated in the first stage
ratio Aspect ratio Consistent with the video aspect ratio (e.g., 9:16）
resolution Image resolution 2k

API return format：

{
  "created": 1700000000,
  "data": [
    { "url": "https://p6-heycan-hgt.byteimg.com/..." }
  ]
}

Step 2: Download the first-frame image locally

IMAGE_URL=$(echo "${RESPONSE}" | jq -r '.data[0].url')
IMAGE_FILE="/tmp/seedance_frame_$(date +%Y%m%d_%H%M%S).png"
curl -sL -o "${IMAGE_FILE}" "${IMAGE_URL}"

Step 3: Show the first-frame image to the user

Report the generation result and let the user proceed to Phase 3 after confirming they are satisfied with the first-frame image.

Phase 3: Seedance 2.0 Video Generation and Download

Step 1: Confirm generation parameters

Parameter Description Default value
API address jimeng-free-api-all service address http://127.0.0.1:8000
SessionID sessionid of the Jimeng platform Environment variable or provided by the user
Model seedance-2.0-fast seedance-2.0-fast
Ratio Video aspect ratio Consistent with the first-frame image
Resolution Video resolution 720p
Duration 4-15 seconds 4
Reference image First-frame image path (generated in Phase 2 or provided by the user) Required

Step 2: Call the Seedance video generation API

Use a local image (multipart/form-data)：

curl -s --max-time 300 -X POST "${API_URL}/v1/videos/generations" \
  -H "Authorization: Bearer ${SESSION_ID}" \
  -F "model=seedance-2.0-fast" \
  -F "prompt=${VIDEO_PROMPT}" \
  -F "ratio=${RATIO}" \
  -F "resolution=${RESOLUTION}" \
  -F "duration=${DURATION}" \
  -F "files=@${IMAGE_FILE}"

When there are multiple images：

curl -s --max-time 300 -X POST "${API_URL}/v1/videos/generations" \
  -H "Authorization: Bearer ${SESSION_ID}" \
  -F "model=seedance-2.0-fast" \
  -F "prompt=${VIDEO_PROMPT}" \
  -F "ratio=${RATIO}" \
  -F "resolution=${RESOLUTION}" \
  -F "duration=${DURATION}" \
  -F "files=@/path/to/image1.jpg" \
  -F "files=@/path/to/image2.jpg"

Important：

  • Seedance 2.0 must upload at least one image, otherwise it returns an error {"code":-2001,"message":"Seedance 2.0 需要至少一张图片"}
  • The API is synchronously blocking and will automatically poll until video generation is complete (usually 60-120 seconds)
  • It is recommended to set the curl timeout to 300 seconds:--max-time 300
  • In the prompt, @1、@2 corresponds to files the upload order of the images in the parameter
  • Authorization headerRequired Bearer prefix, format is Bearer your_sessionid

Step 3: Parse the result

The API returns an OpenAI-compatible format:

{
  "created": 1700000000,
  "data": [
    {
      "url": "https://v3-dreamnia.jimeng.com/...",
      "revised_prompt": "原始提示词"
    }
  ]
}

Step 4: Download the video

VIDEO_URL=$(echo "${RESPONSE}" | jq -r '.data[0].url')
OUTPUT_FILE="seedance_$(date +%Y%m%d_%H%M%S).mp4"
curl -L -o "${OUTPUT_FILE}" "${VIDEO_URL}"
echo "视频已下载到: $(pwd)/${OUTPUT_FILE}"

Step 5: Show the result

Report to the user:

  1. Video generation succeeded/failed
  2. Video file path and size
  3. The first-frame image and storyboard prompt used (to make it easier for the user to fine-tune and retry)

Error handling

API error code

Error Meaning Handling method
code: -2001 Seedance requires at least one image First call the text-to-image API to generate the first frame, then retry
HTTP 200 Success Extract data[0].url
HTTP 401 SessionID expired Prompt the user to obtain it again
HTTP 400 Parameter error Check model/ratio/duration
HTTP 429 Insufficient credits Prompt the user to wait or top up
HTTP 500 Server error Retry or check the service status

Common issues

Issue Solution
Seedance reports an error requiring an image You must first generate the first frame via text-to-image, then call Seedance with the image
Connection refused Check whether the jimeng-free-api-all container is running
Generation timeout Increase curl --max-time, or reduce the duration
Content filtered Modify the prompt to avoid sensitive content
Video URL invalid URLs are time-limited, download promptly
Insufficient credits Daily free credits are limited and refresh the next day

Complete example dialogue

Example 1: Pure text creation (no user image → automatically generate the first frame)

User: Help me generate a video of an orange cat catching crabs by the seaside, 9:16 vertical screen, 4 seconds

Stage 1 - Generate prompt：

First-frame image prompt:

阳光明媚的海边沙滩，一只圆滚滚的橘色猫咪蹲在潮湿的沙滩上，好奇地盯着面前一只小螃蟹，金色阳光照射，海浪轻轻拍打岸边，色彩鲜明活泼，3D动画风格

Video storyboard prompt:

可爱3D动画风格，4秒，9:16竖屏，阳光明媚的海边

@1 作为画面首帧参考

0-1秒：中景，画面中的橘猫好奇地盯着沙滩上的小螃蟹，海浪轻拍岸边
1-3秒：近景跟随，橘猫伸出爪子试探性地去抓螃蟹，螃蟹举起钳子反击，橘猫被吓得往后一跳，表情夸张可爱
3-4秒：中景微拉远，橘猫和螃蟹对峙，背景碧蓝大海和白色浪花

背景音效：欢快俏皮卡通配乐 + 海浪声

Stage 2 - Generate first-frame reference image：

# 1. 调用文生图 API 生成首帧
curl -s --max-time 120 -X POST "${API_URL}/v1/images/generations" \
  -H "Authorization: Bearer ${SESSION_ID}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "jimeng-4.5",
    "prompt": "阳光明媚的海边沙滩，一只圆滚滚的橘色猫咪蹲在潮湿的沙滩上，好奇地盯着面前一只小螃蟹，金色阳光照射，海浪轻轻拍打岸边，色彩鲜明活泼，3D动画风格",
    "ratio": "9:16",
    "resolution": "2k"
  }'

# 2. 下载图片
IMAGE_URL=$(echo "${RESPONSE}" | jq -r '.data[0].url')
curl -sL -o /tmp/seedance_frame.png "${IMAGE_URL}"

Stage 3 - Generate video：

curl -s --max-time 300 -X POST "${API_URL}/v1/videos/generations" \
  -H "Authorization: Bearer ${SESSION_ID}" \
  -F "model=seedance-2.0-fast" \
  -F "prompt=可爱3D动画风格，4秒，9:16竖屏，阳光明媚的海边

@1 作为画面首帧参考

0-1秒：中景，画面中的橘猫好奇地盯着沙滩上的小螃蟹，海浪轻拍岸边
1-3秒：近景跟随，橘猫伸出爪子试探性地去抓螃蟹，螃蟹举起钳子反击，橘猫被吓得往后一跳，表情夸张可爱
3-4秒：中景微拉远，橘猫和螃蟹对峙，背景碧蓝大海和白色浪花

背景音效：欢快俏皮卡通配乐 + 海浪声" \
  -F "ratio=9:16" \
  -F "resolution=720p" \
  -F "duration=4" \
  -F "files=@/tmp/seedance_frame.png"

Example 2: User provides images (skip Stage 2)

User: Use these two images to generate a video of two people dancing

  • Image 1: /home/user/dancer1.jpg
  • Image 2: /home/user/dancer2.jpg

Skip Stage 2, go directly to Stage 3:

curl -s --max-time 300 -X POST "${API_URL}/v1/videos/generations" \
  -H "Authorization: Bearer ${SESSION_ID}" \
  -F "model=seedance-2.0-fast" \
  -F "prompt=电影级写实风格，10秒，4:3画幅，舞台灯光氛围

@1 和 @2 面对面站立，两人开始双人舞，镜头缓慢环绕，动作优雅协调，光影交错" \
  -F "ratio=4:3" \
  -F "resolution=720p" \
  -F "duration=10" \
  -F "files=@/home/user/dancer1.jpg" \
  -F "files=@/home/user/dancer2.jpg"

Storyboard template quick reference

Template 1: Narrative story type

【风格】电影级写实/动画/水墨/科幻风格
【时长】4-15秒
【画幅】16:9

@1 作为画面首帧参考

0-X秒：[镜头运动]，[场景建立]，[主体引入]
X-Y秒：[镜头运动]，[情节发展]，[动作描述]
Y-Z秒：[镜头运动]，[高潮/结尾]

【声音】配乐风格 + 音效 + 对白

Template 2: Product display type

【风格】商业广告/极简/高端/科技感
【时长】4-15秒

@1 作为产品首帧参考

0-X秒：产品展示，运镜环绕/推拉
X-Y秒：产品细节特写，材质/工艺展示
Y-Z秒：品牌落版，slogan展示

【声音】大气恢宏/轻快时尚/科技感配乐

Template 3: Character action type

【风格】根据角色设定（武侠/科幻/现代/奇幻）
【时长】4-15秒

@1 作为角色参考

0-X秒：角色亮相，定格或缓慢展示造型
X-Y秒：核心动作展示（打斗/舞蹈/特技）
Y-Z秒：动作收尾，pose定格

【声音】动作音效 + 氛围配乐

Template 4: Scenery travel type

【风格】电影级纪录片/治愈系/史诗感
【时长】4-15秒

@1 作为场景首帧参考

0-X秒：大景别建立镜头，展示环境全貌
X-Y秒：中景推进，引入人物或细节
Y-Z秒：回到大景别或意境落版

【声音】环境音 + 氛围配乐

Camera language quick reference

Chinese Effect Applicable scenarios
Push-in shot Emphasize the subject, create tension Emotional outburst, detail display
Pull-out shot Show the scene, create a sense of detachment Opening establishment, ending conclusion
Pan shot Show the environment left/right or up/down Environment introduction, character relationships
Tracking shot Parallel movement, follow the subject Walking, chasing
Follow shot Follow the subject as it moves Action follow shooting
Orbit shot 360-degree display of the subject Product display, character reveal
Crane shot From top to bottom or bottom to top Full scene view, display of grandeur
Hitchcock zoom Background compression/stretch effect Suspense, psychological change
Handheld shake Documentary feel/tension Documentary, chasing
One-shot No cuts, continuous shooting Immersiveness, long-form narrative

Atmosphere keyword quick reference

Lighting: backlight, side light, top light, Rembrandt lighting, silhouette, rim light, volumetric light, Tyndall effect Color tone: warm tone, cool tone, high saturation, low saturation, black and white, cyberpunk, retro film Texture: cinematic, documentary style, advertising texture, MV style, oil painting feel, ink painting feel Mood: warm, tense, suspenseful, cheerful, sorrowful, epic, healing, thriller

Notes

  • Seedance 2.0 must provide at least one image, pure text will return an error
  • When there is no user image, a first-frame reference image must first be generated through the text-to-image API
  • Uploading materials containing realistic human faces is not currently supported
  • The total upper limit for mixed input is 12 files
  • API calls are synchronous and blocking; generating a video usually takes 60-120 seconds
  • The video download URL is time-limited; it should be downloaded immediately after generation
  • The daily free credits are limited (about 66 credits); both text-to-image and video generation consume credits, so plan reasonably
  • Authorization headerrequires Bearer prefix, in the format Bearer your_sessionid

Output format

The final output should include:

  1. Understanding confirmation: confirm the understood story content
  2. First-frame prompt: the first-frame image description used for text-to-image
  3. Storyboard prompt: the complete prompt used for Seedance video generation (including the @1 reference)
  4. First-frame image: the path of the reference image generated by text-to-image (if the API was called)
  5. Generation parameters：model/ratio/duration
  6. Generation result: the video file path (if the API was called)