create-video-seedance

create-video-seedance

Purpose

Generate a reference-driven product clip with ByteDance Seedance 2.0 (the seedance_2_0 Higgsfield model). Seedance 2.0 currently leads public leaderboards on image-to-video and text-to-video, with native support for image, video, and audio reference inputs in a single pass.

Use this atom when:

• The clip is product-led and you have a reference image (or multiple SKUs) that must persist.
• Complex camera moves are required (push-in, orbit, jib, dolly).
• An audio reference (music, voice, ambient) should drive the clip's pacing.
• You need start-frame + end-frame conditioning.
• Length up to 15 seconds is needed.

Prefer create-video-veo3 for clean cinematic single-shot ads, create-video-kling for dialogue, create-video-grok for X-native memes.

Inputs

• A clear video brief with: product description, camera move, mood, audio direction, optional genre tag.
• Reference media via the medias parameter — supports roles: image, start_image, end_image, video, audio.
• Aspect ratio: one of auto, 21:9, 16:9, 4:3, 1:1, 3:4, 9:16.
• Duration: 4 to 15 seconds.
• Resolution: 480p, 720p (default), or 1080p.
• Mode: std (default) or fast.
• Genre: auto (default), action, horror, comedy, noir, drama, epic.
• Output directory or test-run directory.

Workflow

1. Read the brief and confirm product description, camera move, mood, and any reference media are present.
2. Compose the prompt. Use skills/atoms/video-generation/_shared/build_animation_prompt.py — it encodes the character-action-verb-led rule learned in the Coinbase music-video debut. The right shape is <character> <ACTION-VERB><object>. <camera/secondary motion>. <style>. NEGATIVE: <cues>. Camera-first prompts ("subtle push-in, slow parallax, [tableau]") produce zoom-on-a-still output that reads as 17 separate clips, not one continuous animation. Lead with the subject + verb.
3. Upload all reference media via mcp__higgsfield__media_upload, confirm with mcp__higgsfield__media_confirm, and assemble the medias list with explicit roles.
4. Call mcp__higgsfield__generate_video with model="seedance_2_0", prompt, aspect_ratio, duration, resolution, mode, genre, and medias.
5. Poll mcp__higgsfield__job_status until complete; download the resulting MP4.
6. Write manifest.json with provider: "Higgsfield", model_or_tool: "seedance_2_0", output paths, status, errors, warnings.
7. Run the verifier in tests/verifier.md.

Output

• Primary: one MP4 clip at the requested aspect ratio, duration, and resolution.
• manifest.json matching the standard schema in TESTING_RULES.md.
• verification.md naming the checks performed.

Quality Checks

• File exists, plays end-to-end, duration within ±0.5s of requested.
• Aspect ratio and resolution match the request.
• Reference identity persists: product matches the supplied reference image.
• Camera move matches the brief direction.
• No severe artifacts.
• Manifest paths resolve to real files.

Failure Modes

• Higgsfield credentials missing or invalid — return manifest with status: "blocked".
• Content moderation block — Seedance has aggressive moderation. See "Content-filter auto-fallback" below; the recommended response is to retry on veo3_1_lite before prompt-rephrasing, because Veo's filter is materially looser on the false-positive surfaces.
• Reference image too small or wrong aspect — Higgsfield prefers ≤4096px on the longest edge; warn and resize before retry.
• Beginner glitch on first try — if the output looks malformed, suggest tightening the camera move and adding a stronger reference image rather than blindly regenerating.
• China data jurisdiction — flag in manifest warnings for clients with data residency constraints.
• Audio reference role rejected — confirm the audio file is uploaded with role audio, not start_image.
• generate_audio unsupported — Seedance 2.0 does not accept this param; use Wan 2.7 or Veo 3.1 if native audio synthesis is required.

Content-filter auto-fallback (Higgsfield path)

Seedance's job_status will sometimes return failed with failure_reason of ip_detected or nsfw on inputs that are clearly safe. Observed false-positive surfaces (Coinbase debut, 2026-05):

Trigger	Example that fired it
ip_detected	Felt-sculpted award statuettes that resemble Oscar / trophy shapes
nsfw	The words "intimate", "leans toward", "presses against" in prose-style prompts (even in non-bodily context)
nsfw	Generic interior scenes (bedroom, hotel room) without any explicit content
nsfw	Close framing of a clothed character lying down or reclining

Recommended behavior in agent loop:

1. Call mcp__higgsfield__generate_video with model="seedance_2_0".
2. Poll job_status. If it returns failed AND failure_reason ∈ {ip_detected, nsfw}:
   • Immediately retry the same job spec with model="veo3_1_lite" (or veo3_1 if quality matters more than cost).
   • Keep the same prompt, medias, aspect_ratio, duration. Veo accepts the same shape.
   • Log the fallback to the project's .fallback-events.jsonl via the orchestrator's scripts/generate_with_fallback.py event-log helper.
3. If Veo also fails on the same prompt with a content-filter reason, the prompt is the issue — escalate to the operator. Do not silently rotate to a third model.
4. If job_status=failed for non-content reasons (rate-limit, invalid input, gateway error), use the existing FAL fallback path documented below — Veo is for content filters specifically, FAL is for Higgsfield-side outages.

The operator can suppress this auto-fallback by passing <provider_override>=higgsfield to the orchestrator. The atom itself does not gate on this — the orchestrator's create-clips or the consuming molecule (e.g. molecules/create-cartoon-music-video) reads the override and decides.

This pattern is duplicated for create-video-veo3 in the reverse direction (Veo → Seedance) only for ip_detected; Veo's NSFW filter is permissive enough that the reverse case is rare.

FAL fallback

When the Higgsfield path fails, the orchestrator's generate_with_fallback.py router calls the sibling FAL script:

python3 skills/atoms/video-generation/create-video-seedance/scripts/generate-fal.py \
  --prompt "..." \
  --output /path/to/clip.mp4 \
  --start-image /path/to/keyframe.png \
  [--end-image /path/to/end-frame.png] \
  --aspect-ratio 9:16 \
  --duration 5 \
  --resolution 720p

⚠️ Version delta: Higgsfield ships Seedance 2.0; FAL's latest as of 2026-05 is Seedance 1.0 Pro. Reference fidelity (multi-SKU consistency, audio-driven pacing) is weaker on v1.0. The fallback's meta.json includes version_caveat; surfaces to the operator on fallback fire.

FAL does NOT currently expose:

• Audio reference media (Seedance 1.0 on FAL is image-only)
• genre tag
• mode: "fast" variant

If any of those are required for the scene, the operator should pin <provider_override>=higgsfield and retry rather than accept the FAL Seedance output.