video-multimodal-generate

Video Multimodal Generate

Overview

Call meitu video-multimodal-generate Perform multimodal hybrid-driven video generation based on any combination of audio/video/image/text. Suitable for pairing music with video, referencing video camera movement/rhythm, and multi-material hybrid generation.

Not handled: text-only video generation, image-only video generation (no video and no audio), motion transfer.

API Mapping

DAG tool	Backend API
video-multimodal-generate	video_multimodal_generate

Dependencies

• meitu-cli: >=2.0.6
• Credentials：CONFIG AKSK → meitu tools update; EXEC AKSK → run command
• Environment variables：MEITU_OPENAPI_TOOL_TASK_MODE=command

Path alias:$VISUAL = {OPENCLAW_HOME}/workspace/visual/

Core Workflow

Preflight → Execute → Deliver

Preflight

1. meitu --version ≥ 2.0.6；meitu auth verify --json
2. Confirm Config Phase has been run;MEITU_OPENAPI_TOOL_TASK_MODE=command
3. output_dir resolution: openclaw.yaml → ./output/ | else → $VISUAL/output/video-multimodal-generate/；mkdir -p {output_dir}

Execute

Trigger signals/routing

Trigger conditions (any one holds):

• User uploads audio → must use this tool (unique capability)
• Video only, no image → camera movement/style/rhythm reference
• User wants to reference the video's camera movement/style/rhythm (not imitate human body motion)
• Complex multi-material mix (image + video + audio)

Decision order:

1. At least one type of multimedia material;prompt missing → ask follow-up
2. Total materials ≤12 (images + videos + audio combined)
3. Unique route → video_multimodal_generate

Has image + video but intent unclear (camera movement vs motion imitation) → default to this tool.

Parameter definitions

Parameter	Type	Required	Range	Default	Description
prompt	STRING	Yes	--	--	Text description; if missing → "Please describe the video content you want to generate"
video_duration	NUMBER	Yes	4–15 / -1 (smart)	-1	Video duration (seconds)
ratio	STRING	Yes	adaptive/16:9/4:3/1:1/3:4/9:16/21:9	adaptive	Aspect ratio
image_list	ARRAY	No	≤9 images	--	List of image URLs
reference_video_list	ARRAY	No	≤3, total duration ≤15s	--	List of reference video URLs
reference_audio_list	ARRAY	No	≤3, total duration ≤15s	--	List of audio URLs
sound	BOOL	No	true / false	true	Whether to generate audio
resolution	STRING	No	480p / 720p	720p	Video resolution

Provide at least one type of multimedia material; images + videos + audio combined ≤12.

Tool invocation

meitu video-multimodal-generate \
  --skill_name skill_video-multimodal-generate \
  --prompt "{prompt}" \
  --video_duration -1 \
  --ratio adaptive \
  --image_list '["{url1}"]' \
  --reference_video_list '["{video_url}"]' \
  --reference_audio_list '["{audio_url}"]' \
  --sound true --resolution 720p \
  --json --download-dir {output_dir}

Error fallback

Scenario	Handling
prompt missing but materials present	Prompt "Please describe the video content you want to generate"
Total materials >12	Prompt input limit, suggest reducing materials
video_duration conflicts with audio duration	Prioritize matching audio duration, inform the user
Some materials inaccessible	Silently filter, continue execution with remaining materials
Invocation failed	Downgrade model version and retry once → if still failing, return error
Timeout	Reduce duration / size and retry once → if still failing, return error
Content compliance interception	Return the compliance prompt directly, do not retry
No multimedia assets at all	Not within the scope of this tool

Deliver

Parse --json → downloaded_files[0].saved_path or media_urls[0]。

mv {file} {output_dir}/{YYYY-MM-DD}_{name}_multimodal.mp4

Output

• Format: MP4 (default 720p, optional 480p)
• Naming：{YYYY-MM-DD}_{descriptive-name}_multimodal.mp4
• Location: Project → ./output/; one-time → $VISUAL/output/video-multimodal-generate/

Baseline Task ID

t_mt1a3i5n7bd3dd7a38-c263-48f7-ac18-bb8573c364b2