byted-mediakit-tools

Note: If the host injects ARK_SKILL_API_BASE / ARK_SKILL_API_KEYinto the environment (for example, for other Skills to go through the SkillHub gateway), it is independent of this Skill's AMK_API_KEY、ARK_API_KEY(video understanding)are independent of each other, do not confuse them.

⚠️ Strictly execute: Audio and video trimming, concatenation, audio extraction, and audio and video synthesis will automatically select the execution backend: when the cloud environment is complete and the input is a URL, use AMK cloud; when the required cloud configuration/dependencies are missing, or the input is a local file path, automatically use local FFmpeg. Video flip, speed adjustment, adding subtitles, adding watermarks, and transcoding are local native commands and always use local FFmpeg. Do not ask users for cloud environment variables for these local capabilities.

<SKILL_DIR> is the directory where byted-mediakit-tools is located. The 链接仅供下载，不支持播放能力 禁止修改任何返回数据信息returned by the current method, such as play_url 、request_id 、task_id , etc. When the user explicitly states that re-execution is required: except for understand_video_content , methods need to generate a new client_token(do not reuse the previous client_token）, to avoid hitting the idempotent result from last time

Volcengine AI MediaKit audio and video processing toolkit

Overview

This toolkit supports the following audio and video processing capabilities. Capabilities marked as “Cloud & Local” automatically select the execution backend: when the cloud configuration is complete and the input is a URL, use AMK cloud; when the cloud is unavailable or the input is a local path, use local FFmpeg.

Capability	Supported scope	Description
Video understanding	Cloud	AI analyzes video content and generates natural language descriptions
Audio and video trimming	Cloud & Local	Precisely trim audio or video duration
Audio and video concatenation	Cloud & Local	Concatenate multiple audio or video clips; cloud supports transition IDs, local supports FFmpeg transition names
Audio extraction	Cloud & Local	Extract audio tracks from video
Audio and video synthesis	Cloud & Local	Synthesize or replace video audio tracks; cloud supports duration alignment strategies, local supports mixing/replacing
Image quality enhancement	Cloud	Improve video image quality, resolution, and frame rate
Text-to-video	Cloud	Generate video from images, supporting animation and transitions
Video flip	Local	Horizontal, vertical, or bidirectional flip
Video speed adjustment	Local	Adjust video and audio playback speed
Add subtitles to video	Local	Add SRT/ASS hard subtitles
Add watermark to video	Local	Add images/watermarks, supports position, scaling, and display time
Video transcoding/remuxing	Local	Convert container format or re-encode

Get API Key

To use cloud capabilities, please obtain an API key first; local overlay capabilities will automatically use FFmpeg when the key is missing:

AI MediaKit Console：https://console.volcengine.com/imp/ai-mediakit/
Ark Model and API Key：https://console.volcengine.com/ark/region:ark+cn-beijing/model/detail?Id=doubao-seed-1-8

Quick Start

1. Environment Configuration

In <SKILL_DIR>/.env , configure the environment variables (a template will be automatically created on first use):

# AMK API Key（云端能力需要；本地回退不需要）
AMK_API_KEY=your_amk_api_key_here
# AMK 环境固定为 prod
AMK_ENV=prod
# 是否启用 client_token 自动注入（用于幂等）
AMK_ENABLE_CLIENT_TOKEN=false
# 方舟 密钥（可选，仅使用视频理解功能时必须配置）
ARK_API_KEY=your_ark_api_key_here
# 方舟 模型ID（可选，仅使用视频理解功能时必须配置）
ARK_MODEL_ID=doubao-seed-1-8

2. Dependency Installation

cd <SKILL_DIR>/scripts
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -U pip
python -m pip install -r requirements.txt

Core Features

Synchronous Capabilities (return results immediately)

Capability	Description
understand_video_content	Video content understanding, use AI to analyze videos and generate natural language descriptions

Asynchronous Capabilities (automatically wait for results by default)

Capability	Description
trim_media_duration	Trim audio/video duration, accurate to the millisecond
concat_media_segments	Concatenate multiple audio/video clips, supports transition effects
extract_audio	Extract audio tracks from videos, supports mp3/m4a formats
enhance_video	Video quality enhancement, supports super-resolution, frame interpolation, etc.
image_to_video	Generate videos from images, supports animation and transitions
mux_audio_video	Audio/video compositing, supports duration alignment

Auxiliary Capabilities

Capability	Description
query_task	Query asynchronous task execution status and results

Local Native Commands

Capability	Description
flip-video / flip_video	Video flipping, defaults to horizontal mirroring
adjust-speed / adjust_speed	Video speed adjustment
add-subtitle / add_subtitle	Add hard subtitles to videos, supports SRT/ASS
add-overlay / add_overlay	Add images/watermarks to videos
transcode	Video transcoding/remuxing

Usage Examples

Video Understanding

./byted-mediakit-tools.sh understand_video_content \
  --video_url "https://example.com/video.mp4" \
  --prompt "总结视频内容" \
  --fps 1

Video Trimming

# 云端：输入为 URL 且 AMK 配置完整
./byted-mediakit-tools.sh trim_media_duration \
  --type video \
  --source "https://example.com/video.mp4" \
  --start_time 0 \
  --end_time 10

# 本地回退：输入为本地文件，或 AMK 配置不完整
./byted-mediakit-tools.sh trim_media_duration \
  --type video \
  --source "/path/to/input.mp4" \
  --start_time 0 \
  --end_time 10 \
  --output "/path/to/output.mp4"

Audio/Video Concatenation

./byted-mediakit-tools.sh concat_media_segments \
  --type video \
  --sources "https://example.com/1.mp4" "https://example.com/2.mp4"

Audio Extraction

./byted-mediakit-tools.sh extract_audio \
  --video_url "https://example.com/video.mp4" \
  --format mp3

Quality Enhancement

./byted-mediakit-tools.sh enhance_video \
  --video_url "https://example.com/video.mp4" \
  --tool_version professional \
  --resolution 1080p

Image to Video

./byted-mediakit-tools.sh image_to_video \
  --images "image_url=https://example.com/1.jpg,duration=3,animation_type=zoom_in" \
           "image_url=https://example.com/2.jpg,duration=3,animation_type=pan_left"

Audio/Video Compositing

./byted-mediakit-tools.sh mux_audio_video \
  --video_url "https://example.com/video.mp4" \
  --audio_url "https://example.com/audio.mp3" \
  --is_audio_reserve false

Local Flip / Speed / Subtitles / Watermark / Transcoding

./byted-mediakit-tools.sh flip-video -i input.mp4 --direction horizontal -o flipped.mp4
./byted-mediakit-tools.sh adjust-speed -i input.mp4 --speed 2.0 -o speed.mp4
./byted-mediakit-tools.sh add-subtitle -i input.mp4 --subtitle sub.srt -o subtitled.mp4
./byted-mediakit-tools.sh add-overlay -i input.mp4 --image logo.png --position top-right -o watermarked.mp4
./byted-mediakit-tools.sh transcode -i input.mov --format mp4 --codec h264 -o output.mp4

Asynchronous Tasks (do not wait for results)

# 使用 --no-wait 立即返回 task_id
./byted-mediakit-tools.sh --no-wait trim_media_duration \
  --type video \
  --source "https://example.com/video.mp4" \
  --start_time 0 \
  --end_time 10

# 查询任务结果
./byted-mediakit-tools.sh query_task --task_id "amk-xxx-xxx"

Response Format

Synchronous Response (Video Understanding)

{
  "status": "success",
  "result": {
    "choices": [
      {
        "role": "assistant",
        "content": "视频内容分析结果..."
      }
    ]
  }
}

Asynchronous Response (automatically waits by default)

{
  "task_id": "amk-tool-extract-audio-xxxxxxxxxxxxxx",
  "duration": 82.454056,
  "play_url": "https://example.vod.cn-north-1.volcvideo.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.mp3?preview=1&auth_key=***",
  "request_id": "20260401xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "status": "completed",
  "task_type": "extract-audio"
}

Asynchronous Response (--no-wait)

{
  "status": "pending",
  "task_id": "amk-xxx-xxx",
  "message": "任务已提交，已跳过等待，可调用 query_task 接口传入 task_id 查询结果",
  "query_example": "./byted-mediakit-tools.sh query_task --task_id amk-xxx-xxx"
}

Error Response

{
  "status": "failed/canceled/timeout",
  "task_id": "amk-xxx-xxx",
  "message": "错误详情"
}

Detailed Documentation

For detailed parameter descriptions of each feature, please refer to the corresponding documents in the reference/ directory:

Capability	Documentation Link
understand_video_content	reference/understand_video_content.md
query_task	reference/query_task.md
concat_media_segments	reference/concat_media_segments.md
enhance_video	reference/enhance_video.md
extract_audio	reference/extract_audio.md
image_to_video	reference/image_to_video.md
mux_audio_video	reference/mux_audio_video.md
trim_media_duration	reference/trim_media_duration.md
Unified Response Format	reference/common_response.md
Local Fallback Strategy	reference/local_fallback.md

Notes

Returned Links: All returned play_url and other links are for download only and do not support direct playback
Idempotency: When re-executing tasks, please ensure a new client_token（AMK_ENABLE_CLIENT_TOKEN=true is generated automatically at that time)
Video Understanding: Using the video understanding feature requires configuring ARK_API_KEY and ARK_MODEL_ID
Timeout Handling: Processing large files may take a long time; it is recommended to use --no-wait with query_task polling
Local Capabilities：trim_media_duration、concat_media_segments、extract_audio、mux_audio_video supports local FFmpeg fallback;flip-video、adjust-speed、add-subtitle、add-overlay、transcode Always use local FFmpeg;enhance_video、image_to_video、understand_video_content、query_task Cloud only