byted-mediakit-process-tools

Note: If the host injects ARK_SKILL_API_BASE / ARK_SKILL_API_KEYinto the environment (for example, for other Skills to use through the SkillHub gateway), it is independent of this Skill's AMK_API_KEY、ARK_API_KEY(video understanding)and should not be confused.

⚠️ Strictly execute: audio-video trimming, concatenation, audio extraction, and audio-video synthesis automatically select the execution backend: when the cloud environment is complete and the input is a URL, use the AMK cloud; when required cloud configuration/dependencies are missing, or the input is a local file path, automatically use local FFmpeg. Video flip, speed adjustment, adding subtitles, adding watermark, and transcoding are local native commands and always use local FFmpeg. Do not ask users for cloud environment variables for these local capabilities.

<SKILL_DIR> is the directory where byted-mediakit-process-tools is located. The current method returns 链接仅供下载，不支持播放能力 禁止修改任何返回数据信息, such as play_url 、request_id 、task_id , etc. When the user explicitly states that re-execution is required: methods other than understand_video_content need to generate a new client_token(do not reuse the previous client_token）, to avoid hitting the idempotent result from last time

Volcengine AI MediaKit audio and video processing toolkit

Overview

This toolkit supports the following audio and video processing capabilities. Capabilities marked as “Cloud & Local” automatically select the execution backend: when the cloud configuration is complete and the input is a URL, use the AMK cloud; when the cloud is unavailable or the input is a local path, use local FFmpeg.

Capability	Support scope	Description
Video understanding	Cloud	AI analyzes video content and generates natural language descriptions
Audio-video trimming	Cloud & Local	Precisely trim audio or video duration
Audio-video concatenation	Cloud & Local	Concatenate multiple audio or video clips; the cloud supports transition IDs, and local supports FFmpeg transition names
Audio extraction	Cloud & Local	Extract the audio track from a video
Audio-video synthesis	Cloud & Local	Synthesize or replace a video audio track; the cloud supports duration alignment strategies, and local supports mixing/replacement
Image quality enhancement	Cloud	Improve video image quality, resolution, and frame rate
Text-to-video	Cloud	Generate video from images, supporting animation and transitions
Video flip	Local	Horizontal, vertical, or bidirectional flip
Video speed adjustment	Local	Adjust video and audio playback speed
Add video subtitles	Local	Add SRT/ASS hard subtitles
Add video watermark	Local	Add image/watermark, supporting position, scaling, and display time
Video transcoding/remuxing	Local	Convert container format or re-encode

Get API Key

To use cloud capabilities, first obtain an API key; local override capabilities will automatically use FFmpeg when the key is missing:

AI MediaKit Console：https://console.volcengine.com/imp/ai-mediakit/
Ark Model and API Key：https://console.volcengine.com/ark/region:ark+cn-beijing/model/detail?Id=doubao-seed-1-8

Quick Start

1. Environment Configuration

In <SKILL_DIR>/.env , configure the environment variables (a template will be automatically created on first use):

# AMK API Key（云端能力需要；本地回退不需要）
AMK_API_KEY=your_amk_api_key_here
# AMK 环境固定为 prod
AMK_ENV=prod
# 是否启用 client_token 自动注入（用于幂等）
AMK_ENABLE_CLIENT_TOKEN=false
# 方舟 密钥（可选，仅使用视频理解功能时必须配置）
ARK_API_KEY=your_ark_api_key_here
# 方舟 模型ID（可选，仅使用视频理解功能时必须配置）
ARK_MODEL_ID=doubao-seed-1-8

2. Dependency Installation

cd <SKILL_DIR>/scripts
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -U pip
python -m pip install -r requirements.txt

Core Features

Synchronous capabilities (return results immediately)

Capability	Description
understand_video_content	Video content understanding, using AI to analyze videos and generate natural language descriptions

Asynchronous capabilities (automatically wait for results by default)

Capability	Description
trim_media_duration	Trim audio/video duration, accurate to milliseconds
concat_media_segments	Concatenate multiple audio/video clips, supporting transition effects
extract_audio	Extract audio tracks from video, supporting mp3/m4a formats
enhance_video	Video quality enhancement, supporting super-resolution, frame interpolation, etc.
image_to_video	Generate video from images, supporting animation and transitions
mux_audio_video	Audio/video compositing, supporting duration alignment

Auxiliary capabilities

Capability	Description
query_task	Query asynchronous task execution status and results

Local native commands

Capability	Description
flip-video / flip_video	Video flipping, horizontal mirroring by default
adjust-speed / adjust_speed	Video speed adjustment
add-subtitle / add_subtitle	Add hard subtitles to video, supporting SRT/ASS
add-overlay / add_overlay	Add images/watermarks to video
transcode	Video transcoding/remuxing

Usage Examples

Video Understanding

./byted-mediakit-process-tools.sh understand_video_content \
  --video_url "https://example.com/video.mp4" \
  --prompt "总结视频内容" \
  --fps 1

Video Trimming

# 云端：输入为 URL 且 AMK 配置完整
./byted-mediakit-process-tools.sh trim_media_duration \
  --type video \
  --source "https://example.com/video.mp4" \
  --start_time 0 \
  --end_time 10

# 本地回退：输入为本地文件，或 AMK 配置不完整
./byted-mediakit-process-tools.sh trim_media_duration \
  --type video \
  --source "/path/to/input.mp4" \
  --start_time 0 \
  --end_time 10 \
  --output "/path/to/output.mp4"

Audio/Video Concatenation

./byted-mediakit-process-tools.sh concat_media_segments \
  --type video \
  --sources "https://example.com/1.mp4" "https://example.com/2.mp4"

Audio Extraction

./byted-mediakit-process-tools.sh extract_audio \
  --video_url "https://example.com/video.mp4" \
  --format mp3

Quality Enhancement

./byted-mediakit-process-tools.sh enhance_video \
  --video_url "https://example.com/video.mp4" \
  --tool_version professional \
  --resolution 1080p

Generate Video from Images

./byted-mediakit-process-tools.sh image_to_video \
  --images "image_url=https://example.com/1.jpg,duration=3,animation_type=zoom_in" \
           "image_url=https://example.com/2.jpg,duration=3,animation_type=pan_left"

Audio/Video Compositing

./byted-mediakit-process-tools.sh mux_audio_video \
  --video_url "https://example.com/video.mp4" \
  --audio_url "https://example.com/audio.mp3" \
  --is_audio_reserve false

Local flip / speed adjustment / subtitles / watermark / transcoding

./byted-mediakit-process-tools.sh flip-video -i input.mp4 --direction horizontal -o flipped.mp4
./byted-mediakit-process-tools.sh adjust-speed -i input.mp4 --speed 2.0 -o speed.mp4
./byted-mediakit-process-tools.sh add-subtitle -i input.mp4 --subtitle sub.srt -o subtitled.mp4
./byted-mediakit-process-tools.sh add-overlay -i input.mp4 --image logo.png --position top-right -o watermarked.mp4
./byted-mediakit-process-tools.sh transcode -i input.mov --format mp4 --codec h264 -o output.mp4

Asynchronous tasks (do not wait for results)

# 使用 --no-wait 立即返回 task_id
./byted-mediakit-process-tools.sh --no-wait trim_media_duration \
  --type video \
  --source "https://example.com/video.mp4" \
  --start_time 0 \
  --end_time 10

# 查询任务结果
./byted-mediakit-process-tools.sh query_task --task_id "amk-xxx-xxx"

Response Format

Synchronous response (video understanding)

{
  "status": "success",
  "result": {
    "choices": [
      {
        "role": "assistant",
        "content": "视频内容分析结果..."
      }
    ]
  }
}

Asynchronous response (automatically wait by default)

{
  "task_id": "amk-tool-extract-audio-xxxxxxxxxxxxxx",
  "duration": 82.454056,
  "play_url": "https://example.vod.cn-north-1.volcvideo.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.mp3?preview=1&auth_key=***",
  "request_id": "20260401xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "status": "completed",
  "task_type": "extract-audio"
}

Asynchronous response (--no-wait)

{
  "status": "pending",
  "task_id": "amk-xxx-xxx",
  "message": "任务已提交，已跳过等待，可调用 query_task 接口传入 task_id 查询结果",
  "query_example": "./byted-mediakit-process-tools.sh query_task --task_id amk-xxx-xxx"
}

Error response

{
  "status": "failed/canceled/timeout",
  "task_id": "amk-xxx-xxx",
  "message": "错误详情"
}

Detailed Documentation

For detailed parameter descriptions of each feature, please refer to the corresponding documents in the reference/ directory:

Capability	Document Link
understand_video_content	reference/understand_video_content.md
query_task	reference/query_task.md
concat_media_segments	reference/concat_media_segments.md
enhance_video	reference/enhance_video.md
extract_audio	reference/extract_audio.md
image_to_video	reference/image_to_video.md
mux_audio_video	reference/mux_audio_video.md
trim_media_duration	reference/trim_media_duration.md
Unified response format	reference/common_response.md
Local fallback strategy	reference/local_fallback.md

Notes

Returned links: All returned play_url and other links are for download only and do not support direct playback
Idempotency: When re-executing tasks, please ensure new client_token（AMK_ENABLE_CLIENT_TOKEN=true is automatically handled)
Video Understanding: Using the video understanding feature requires configuring ARK_API_KEY and ARK_MODEL_ID
Timeout Handling: Processing large files may take a long time; it is recommended to use --no-wait with query_task polling
Local capabilities：trim_media_duration、concat_media_segments、extract_audio、mux_audio_video support local FFmpeg fallback;flip-video、adjust-speed、add-subtitle、add-overlay、transcode Always use local FFmpeg;enhance_video、image_to_video、understand_video_content、query_task Cloud only