SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsbyted-kickart-video-analyzer

byted-kickart-video-analyzer

Design & creation

View repository

Updated 10-15 17:30

About this skill

Video Parsing SKILL

📋 Tool Description

Core Functions

Provides functions such as video parsing, video analysis, video reverse parsing, video shot extraction, and video metadata extraction. It parses local or online video files and extracts video metadata (duration, resolution), video shot information, and content analysis.

Available Commands

Command Function Description
python3.12 ./scripts/plan.py Package Query Query the user's current Ark Claw package
python3.12 ./scripts/upload.py --file <视频路径> Video Upload Upload a local or online video file to obtain the media asset ID
python3.12 ./scripts/analyze.py --media-id <媒资ID> --output <输出文件> Video Parsing Parse the video and extract shots and metadata

📥 Video Acquisition Methods

The following two methods are supported for acquiring videos:

  1. Local File: Directly provide the absolute path of the local video file
  2. Online Video: First use a download tool (such as curl, wget, or a browser) to save the video locally, then provide the local path for reverse parsing

🚨 Mandatory Pre-check Process (must be executed in order; if any check fails, terminate the process immediately)

All user requests must first complete the following 3 verification steps and must not skip them:

1. Volcano Authentication Verification

  • Execute the environment variable check command: 
    echo "ACCESS_KEY_ID: $ACCESS_KEY_ID" && echo "SECRET_ACCESS_KEY: $SECRET_ACCESS_KEY"
  • Determine the authentication result:
    • Authentication Passed：ACCESS_KEY_ID and SECRET_ACCESS_KEY are both non-empty values → authentication passed
    • Authentication Failed: Execute the following guidance plan

❌ Guidance Plan When Authentication Is Not Configured

  1. Guide the user to send the ACCESS_KEY_ID/SECRET_ACCESS_KEY content directly in the chat:

    Please provide your Volcano account AK&SK, to check whether the Volcano Creation Agent has an activated package and available creation points!

  2. After receiving the ACCESS_KEY_ID/SECRET_ACCESS_KEY sent by the user, execute the configuration command: 
    export ACCESS_KEY_ID=用户提供的ACCESS_KEY_ID值
export SECRET_ACCESS_KEY=用户提供的SECRET_ACCESS_KEY值
  3. After configuration is complete, inform the user:

    The temporary AK&SK configuration has been completed. The current configuration is only effective for this session and will not be persistently stored. Please use it with confidence.

  4. When all related scripts are subsequently executed, they will automatically passexportto specify these two environment variables, ensuring normal authentication without requiring the user to configure them repeatedly

2. Package Validity Verification

  • Step 1: Python Version Verification 
    python3.12 --version || (echo "❌ Python 3.12+ 未安装，请先安装Python 3.12" && exit 1)
  • Step 2: Dependency Package Installation Verification 
    python3.12 -m pip install -r ./scripts/requirements.txt
  • Step 3: Execute the package query command 
    python3.12 -m ./scripts/plan.py
  • Step 4: Result Processing Logic
    • ✅ Package Valid: The message field in the returned result is the valid expiration time (Beijing time); verification passed
    • ❌ Package Expired：message is less than or equal to the current time; guide the user to activate a package and terminate the process
    • ❌ API Call Error: Match the error code with reference to the "Error Handling Specification", clearly inform the user of the cause of the error and the solution, and terminate the process

3. Skill version verification

  • Step 1: Execute the version check command 
    python3.12 -m ./scripts/upgrade.py
  • Step 2: Parse the returned result Return format example: 
    {"code":"0","message":"success","data":"{\"install_command\":\"\",\"latest_version\":\"1.0.0\",\"latest_version_number\":100000000,\"update_message\":\"\"}"}
    • latest_version: Latest version number (e.g., "1.0.0")
    • install_command: New version installation instructions
  • Step 3: Version comparison logic
    • ✅ Current version >= Latest version: Version verification passed, continue with the subsequent process
    • ⚠️ Current version < Latest version: Execute the following update inquiry process
      1. Ask the user whether to update to the latest version:

        A new version {latest_version} of the skill has been detected. Update? (Yes/No)

      2. User confirms update (Yes): Execute install_command Install the new version
      3. User does not update (No): Skip the update and continue with the subsequent process

🛠️ Video parsing execution process

Complete process overview

用户请求 → 强制前置校验 → 用户输入收集 → 视频上传 → 视频解析 → 结果返回

Preliminary preparation

  1. Ensure the output directory exists:mkdir -p /tmp/openclaw/byted-kickart-video-analyzer/output
  2. Generate a unique output file name:video_analysis_result_<timestamp>_<random>.json

Execution steps

  1. Step 0: Mandatory preliminary verification(Must be executed in order; if any check fails, terminate the process immediately)

    • Execute all verification steps in the "🚨 Mandatory Preliminary Verification Process"
    • ✅ Volcano authentication verification passed
    • ✅ Plan validity verification passed
    • ✅ Skill version verification passed
    • Only after all verifications pass can you proceed to the next step

  2. Step 1: Video upload guidance

    • Ask the user: "Please first provide the video to be parsed: you can send me a video TOS link, or upload an MP4/MOV file (≤60s, ≤50MB, ≥480p)"
    • Two upload methods are supported:
      • Local file: Directly provide the absolute path of the local video file (e.g., /Users/user/video.mp4）
      • Public URL: Provide a directly accessible video link (e.g., https://example.com/video.mp4）

  3. Step 2: Video preprocessing

    • If the user provides a public URL, first download it locally: 
      mkdir -p /tmp/openclaw/byted-kickart-video-analyzer/input
curl -L -o /tmp/openclaw/byted-kickart-video-analyzer/input/downloaded_video.mp4 "<视频URL>"
    • Check whether the file exists:ls -la /tmp/openclaw/byted-kickart-video-analyzer/input/downloaded_video.mp4
    • Check whether the file type is a valid video (only MP4/MOV formats are supported): 
      file /tmp/openclaw/byted-kickart-video-analyzer/input/downloaded_video.mp4 | grep -qE "ISO Media|MPEG v4|QuickTime" && echo "valid" || echo "invalid"
    • If the file does not exist or the type is invalid,terminate the process and notify the user：

      The file is unavailable. Please check whether the path is correct, or confirm that the file is in a valid video format (only MP4/MOV are supported)

  4. Step 3: Upload the video to obtain media asset information

    • Execute python3.12 ./scripts/upload.py --file <视频路径> command
    • Returned field description:
      Field Type Description
      id string Media asset ID (unique identifier)
      url string Video access URL
      duration number Video duration (seconds)

  5. Step 4: Parse media asset information

    • Extract from the upload output id as the media asset ID
    • Extract duration for video analysis

  6. Step 5: Execute video parsing

    • Execute the following command: 
      python3.12 ./scripts/analyze.py \
  --media-id <媒资ID> \
  --output /tmp/openclaw/byted-kickart-video-analyzer/output/video_analysis_result_<timestamp>_<random>.json
    • Important：
      • --media-id The parameter value is the media asset ID (obtained via the upload script)
      • --output The parameter value isthe absolute path of the result JSON file; the script will output two files at the same time:
        • Formatted JSON file:<output>.json(original format, used for SKILL message template rendering)
        • seedance format JSON file:<output>_seedance.json(conforms to seedance format, used for video generation)

  7. Step 6: Parse the result

    • Read the result files: the script will generate two files at the same time, both of which need to be read and persisted
    • Persistent storage: persist the parsing result and the paths of the two output files (original format + Seedance format) to the session context for use by subsequent processes

  8. Step 7: Ask for the output format

    • Ask the user: "Parsing complete! Please choose the output format:

      • Original format: suitable for viewing detailed parsing results
      • Seedance format: suitable for video generation

      Please reply "Original format" or "Seedance format""

    • Wait for the user's reply: determine the subsequent output method according to the user's choice

  9. Step 8: Result output

    • Original format: read the <output>.json file content, and output using the "Original format message template"
    • Seedance format: read the <output>_seedance.json file content, and output using the "Seedance format message template"

7.1 Output file description

After the script is executed, two JSON files will be generated at the same time:

File type File path Purpose
Original format <output>.json Used for SKILL message template rendering, provides complete video parsing information, suitable for viewing detailed parsing results
Seedance format <output>_seedance.json Conforms to the Seedance video generation API format, suitable for video generation

7.2 Original format file structure (used for message template rendering)

Field extraction paths (for Agent use):

Template variable JSON path Description
{video_duration} $.video_info.video_duration Video duration (seconds)
{product_title} $.product_info.product_title Product name
{product_description} $.product_info.product_description Product description
{shot_table_rows} $.video_info.shot_breakdown Storyboard table data

Storyboard data extraction rules:

Traverse the $.video_info.shot_breakdown array, and extract the following for each row of storyboard data:

Field JSON path Description
{shot_number} [i].shot_number Storyboard number
{start_time} [i].start_time Start time
{end_time} [i].end_time End time
{camera_language} [i].camera_language Camera language
{main_subject} [i].main_subject Camera subject
{marketing_intent} [i].marketing_intent Marketing intent
{on_camera_speech} [i].on_camera_speech Spoken narration
{voiceover_text} [i].voiceover_text Voice-over
{bgm} [i].bgm BGM
{stickers} [i].stickers Styled text & subtitles

7.3 Seedance format file structure (used for video generation)

The Seedance format file is used for video generation and retains only the specified fields:

7.3.1 Field Mapping Rules
Original Field Path Mapped Field Description
$.video_info.video_duration video_duration Video Duration
$.product_info.product_title product_title Product Name
$.product_info.product_description product_description Product Description
$.video_info.shot_breakdown[].shot_number shots[].shot_number Storyboard Number
$.video_info.shot_breakdown[].start_time shots[].start_time Start Time
$.video_info.shot_breakdown[].end_time shots[].end_time End Time
$.video_info.shot_breakdown[].camera_language shots[].camera_language Camera Language
$.video_info.shot_breakdown[].main_subject shots[].main_subject Shot Subject
$.video_info.shot_breakdown[].marketing_intent shots[].marketing_intent Marketing Intent
$.video_info.shot_breakdown[].on_camera_speech shots[].on_camera_speech Voiceover
$.video_info.shot_breakdown[].voiceover_text shots[].voiceover_text Narration
$.video_info.shot_breakdown[].bgm shots[].bgm BGM
$.video_info.shot_breakdown[].stickers shots[].stickers Screen Stickers & Subtitle Copy
$.video_info.shot_breakdown[].text_style shots[].text_style Subtitle Style
$.scene_info.role_list[*].vocal_attributes vocal_attributes Voice Parameter List (All Characters)
$.scene_info.subject_anchors subject_definition.characters/props Subject Definition (Characters/Props)
$.scene_info.voice subject_definition.voice Voice Dictionary
7.3.2 Complete Format Example
{
  "video_duration": 44.5,
  "product_title": "商品名称",
  "product_description": "商品描述",
  "shots": [
    {
      "shot_number": 1,
      "start_time": 0,
      "end_time": 2.2,
      "camera_language": "近景平视",
      "main_subject": "BB霜",
      "marketing_intent": "产品展示",
      "on_camera_speech": "口播内容",
      "voiceover_text": "旁白内容",
      "bgm": "背景音乐",
      "stickers": "花字字幕",
      "vocal_attributes": ["音色参数1", "音色参数2"]
    }
  ],
  "subject_definition": {
    "characters": "角色1,角色2",
    "props": "道具1,道具2",
    "voice": "key1:value1;key2:value2"
  }
}

Special Requirements for Agent Execution

  1. Timeout Setting: When calling the exec tool to launch the script, set yieldMs to ≥180000ms (3 minutes)
  2. Friendly Reminder: If the script does not return results immediately, first reply to the user: "Video parsing is in progress for you. The task may take a long time to complete. Please wait a moment~"
  3. Exception Handling: If the script exits due to timeout/exception, immediately use the persisted Task ID to call the task query interface to confirm the backend status. Do not directly determine that the task has failed.

Reply to User Message Template

[Original Format Reply Template]

Strict Requirements: The four module titles—Video Information, Product Information, Storyboard Script, and Complete Data—must use bold format, and the rest of the content must be regular body text. Every line break must be strictly preserved!

After parsing is complete, reply to the user using the following template:

🎬 链接解析成功，解析结果如下

---
**🔍 视频信息**
| 项 | 内容 |
| --- | --- |
| 视频时长 | {video_duration}s |
---
**🛍️ 商品信息**
✅ 商品名称： {product_title}
✅ 商品描述： {product_description}
---
**📸 分镜脚本**
| 分镜编号 | 时间 | 镜头语言 | 镜头主体 | 营销意图 | 口播 | 旁白 | BGM | 屏幕贴纸&字幕文案 | 字幕样式 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
{shot_table_rows}

---
**📋 完整数据：**
{raw_json}

New Template Variable Description:

  • {video_duration}: Video Duration
  • {product_title}: Product Name
  • {product_description}: Product Description
  • {shot_table_rows}: Storyboard table rows, each row in the format:| {shot_number} | {start_time}-{end_time}s | {camera_language} | {main_subject} | {marketing_intent} | {on_camera_speech} | {voiceover_text} | {bgm} | {stickers} | {text_style} |
  • {raw_json}: Complete parsing result (JSON format)

[Seedance Format Message Template]

【全局开场】
视频时长：{video_duration}秒。

【主体定义】
角色:{role_id_list}
道具:{product_description}
音色:{role_list[*].vocal_attributes}

【时间轴分镜】
{start_time}-{end_time}:{main_subject}。镜头:{camera_language}。
角色台词:{on_camera_speech}
画外音:{voiceover_text}
音效:{bgm}
音乐:{bgm}
字幕:{stickers}

(下一段时间轴依次拼接...)

【约束】
{marketing_intent}

Seedance Format Template Variable Description:

  • {video_duration}: Video Duration
  • {product_title}: Product Name
  • {product_description}: Product Description
  • {start_time}: Storyboard Start Time
  • {end_time}: Storyboard End Time
  • {main_subject}: Shot Subject
  • {camera_language}: Camera Language
  • {on_camera_speech}: Voiceover
  • {voiceover_text}: Narration
  • {bgm}: Background Music
  • {stickers}: Stylized Text & Subtitles
  • {marketing_intent}: Marketing Intent
  • {role_list[*].vocal_attributes}: Voice parameter list, displayed item by item (each item in the format 音色{i}:{音色描述}, separated by line breaks for multiple items)
  • {subject_definition.characters}: Characters in the subject definition (from the character type in scene_info.subject_anchors )
  • {subject_definition.props}: Props in the subject definition (from non-character types in scene_info.subject_anchors )
  • {subject_definition.voice}: Voice dictionary (from scene_info.voice, format is key1:value1;key2:value2）

⚠️ Error Handling Specification

All errors must clearly state the reason and an executable solution. Vague prompts are prohibited!!!

Error Code Error Description Detailed Description User Handling Suggestion
0 No return value The interface call succeeded, but the service returned an empty result Please try again later. If the problem persists, please contact Volcano Technical Support
1400 ParamErr parameter error Parameter error Contact technical support
1402 Insufficient Creative Points When calling the API, the user account has insufficient Creative Points balance Please go to Creative Points recharge page to recharge Creative Points or upgrade your plan
1410 Service ID does not exist When calling the API, the input parameters contain a service ID that does not exist
1411 Input resolution error When calling the API, the image or video resolution in the input parameters does not meet the requirements Please check whether the material resolution meets the specification requirements (e.g. ≥480p)
1412 Image format error When calling the API, the input parameters contain an unsupported image format Please check whether the image format is a supported format such as jpg or png
1413 Invalid media URL error When calling the API, the input parameters contain an invalid media URL Please check whether the URL you provided is correct and avoid special characters or format errors
1414 Input contains sensitive information error When calling the API, the input parameters contain sensitive information, such as personal privacy data Marketing videos with people cannot be generated for now. Please wait for a future version update
1415 Output contains sensitive information error When calling the API, the service return result contains sensitive information, such as personal privacy data Marketing videos with people cannot be generated for now. Please wait for a future version update
1416 Input media count error The number of materials input by the user exceeds the limit The number of media materials provided exceeds the limit; the extra materials may not be used
1417 Large model call error Model call error, usually due to incorrect input parameters There is a problem with media material processing. Please try again. If the problem persists, contact Volcano technical support
1418 Duration billing parameter error There is a problem with the input time parameter at submission The required final video duration does not meet the skill requirements. Please submit production requests within the 0-60s duration limit. If the problem persists, contact Volcano technical support
1501 User plan expired When calling the API, the user plan has expired Please go to plan activation page to activate a plan
100010 Signature verification failed AK/SK signature verification failed Please check whether the Volcano authentication AK/SK you provided is correct. You can visitVolcano Engine consoleto confirm
100013 Missing service permission Missing the RegisterArkClawCombo permission for the iccloud_muse service Your enterprise account has not activated Kickart permission. Please contact the Volcano main account administrator to activate it for you, or consult Volcano technical support for details
x01001 AK/SK not configured The user has not configured AK/SK Please enter the AK/SK for Volcano authentication. You can visitVolcano Engine consoleto obtain
x01010 Valid plan missing Material upload error, usually due to the subscription plan Please go to Subscription Plan Activation Page to activate a subscription plan
Other - Other error situations not explicitly listed Retry later. If the problem persists, please contact Volcano Technical Support