Overview
This skill provides comprehensive audio and video processing capabilities based on ffmpeg, including battle-tested commands and workflows, suitable for common multimedia tasks, platform-specific optimization, and best practices for quality and file size management.
Version: 1.0.0 Requirements: ffmpeg >= 4.0, ffprobe (optional but recommended)
When the user mentions video or audio processing tasks, format conversion, social media optimization, or multimedia editing, Claude should use this skill.
Use Cases
Use this skill when the user needs the following operations:
- Video format conversion (MP4, WebM, MOV, etc.)
- Resolution scaling or aspect ratio adjustment
- Create GIFs from videos
- Audio extraction or format conversion
- Video editing (trimming, merging, speed changes, rotation)
- Subtitle processing (hard subtitles, soft subtitles, subtitle extraction)
- Video compression or optimization
- Platform-specific formatting (YouTube, Instagram, TikTok, Twitter)
- Thumbnail or frame extraction
- Batch processing of audio and video files
Prerequisites
Before using this skill, please ensure ffmpeg is installed:
# macOS
brew install ffmpeg
# Ubuntu/Debian
sudo apt-get install ffmpeg
# Windows(使用 Chocolatey)
choco install ffmpeg
Verify installation:
ffmpeg -version
Supported Operations
1. Format Conversion
Convert between different video formats and use optimized settings.
MP4 to WebM:
ffmpeg -i input.mp4 -c:v libvpx-vp9 -crf 30 -b:v 0 -c:a libopus output.webm
MOV to MP4:
ffmpeg -i input.mov -c:v libx264 -c:a aac -strict experimental output.mp4
Any format to MP4 (universal compatibility):
ffmpeg -i input.* -c:v libx264 -preset medium -crf 23 -c:a aac -b:a 128k output.mp4
2. Resolution Adjustment
Resize videos while maintaining the aspect ratio.
Scale to 720p:
ffmpeg -i input.mp4 -vf scale=-1:720 -c:a copy output_720p.mp4
Scale to 1080p:
ffmpeg -i input.mp4 -vf scale=-1:1080 -c:a copy output_1080p.mp4
Scale to a specified width (height auto-adjusted):
ffmpeg -i input.mp4 -vf scale=1280:-1 -c:a copy output.mp4
Scaling with padding (black bar padding/letterbox mode):
ffmpeg -i input.mp4 -vf "scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2" output.mp4
3. GIF Creation
Create high-quality GIFs from videos and optimize file size.
Basic GIF (10 frames per second):
ffmpeg -i input.mp4 -vf "fps=10,scale=480:-1:flags=lanczos" output.gif
High-quality GIF (using a palette):
# 生成调色板
ffmpeg -i input.mp4 -vf "fps=10,scale=480:-1:flags=lanczos,palettegen" palette.png
# 使用调色板创建 GIF
ffmpeg -i input.mp4 -i palette.png -filter_complex "fps=10,scale=480:-1:flags=lanczos[x];[x][1:v]paletteuse" output.gif
Create a GIF from a specified time range:
ffmpeg -ss 00:00:10 -t 5 -i input.mp4 -vf "fps=10,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" output.gif
4. Audio Operations
Extract, convert, and process audio streams.
Extract audio as MP3:
ffmpeg -i input.mp4 -vn -acodec libmp3lame -q:a 2 output.mp3
Extract audio as WAV:
ffmpeg -i input.mp4 -vn -acodec pcm_s16le -ar 44100 -ac 2 output.wav
Convert audio format:
ffmpeg -i input.wav -c:a aac -b:a 192k output.m4a
Add background music:
ffmpeg -i video.mp4 -i music.mp3 -c:v copy -c:a aac -map 0:v:0 -map 1:a:0 -shortest output.mp4
Mix audio (overlay):
ffmpeg -i video.mp4 -i music.mp3 -filter_complex "[0:a][1:a]amix=inputs=2:duration=first" -c:v copy output.mp4
5. Video Editing
Trim, concatenate, and modify videos.
Trim video:
# 从第 10 秒到第 30 秒
ffmpeg -i input.mp4 -ss 00:00:10 -to 00:00:30 -c copy output.mp4
# 基于时长(从第 5 秒开始,截取 10 秒)
ffmpeg -i input.mp4 -ss 00:00:05 -t 10 -c copy output.mp4
Concatenate videos:
Choose the method based on format and compatibility:
Method 1: Concat protocol (recommended — no temporary files needed)
# 适用于 MPEG 格式:.ts、.mpg、.mpeg、.mp3、.aac 等
# 直接拼接,无需创建列表文件
ffmpeg -i "concat:file1.mp3|file2.mp3|file3.mp3" -c copy output.mp3
ffmpeg -i "concat:video1.ts|video2.ts|video3.ts" -c copy output.ts
# 支持格式:TS、MPEG-1、MPEG-2、MP3、AAC
# 不支持:MP4、MOV、MKV(请使用方法二)
Method 2: Concat demuxer (suitable for MP4, MOV, MKV)
# 使用进程替换避免临时文件
ffmpeg -f concat -safe 0 -i <(printf "file '%s'\n" video1.mp4 video2.mp4 video3.mp4) -c copy output.mp4
# 如果 shell 不支持进程替换:
printf "file '%s'\n" video1.mp4 video2.mp4 video3.mp4 > list.txt
ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4
rm list.txt
Method 3: Concat filter (use when re-encoding is acceptable)
# 当视频编码/分辨率不同时使用
ffmpeg -i video1.mp4 -i video2.mp4 -i video3.mp4 \
-filter_complex "[0:v][0:a][1:v][1:a][2:v][2:a]concat=n=3:v=1:a=1[v][a]" \
-map "[v]" -map "[a]" output.mp4
Format selection guide:
.mp3、.aac、.ts、.mpg、.mpeg→ use concat protocol (Method 1).mp4、.mov、.mkv→ use concat demuxer (Method 2)- Different encodings/resolutions → use concat filter (Method 3)
Speed up/slow down:
# 2 倍速
ffmpeg -i input.mp4 -filter:v "setpts=0.5*PTS" -an output.mp4
# 0.5 倍速(慢动作)
ffmpeg -i input.mp4 -filter:v "setpts=2.0*PTS" output.mp4
Rotate video:
# 顺时针旋转 90 度
ffmpeg -i input.mp4 -vf "transpose=1" output.mp4
# 旋转 180 度
ffmpeg -i input.mp4 -vf "transpose=2,transpose=2" output.mp4
6. Subtitle Processing
Add, extract, or burn in subtitles.
Burn subtitles into video (hard subtitles):
ffmpeg -i input.mp4 -vf subtitles=subtitles.srt output.mp4
Add soft subtitles:
ffmpeg -i input.mp4 -i subtitles.srt -c copy -c:s mov_text output.mp4
Extract subtitles:
ffmpeg -i input.mp4 -map 0:s:0 subtitles.srt
7. Thumbnail Extraction
Extract frames from a video as images.
Extract a single frame at a specified time:
ffmpeg -i input.mp4 -ss 00:00:05 -vframes 1 thumbnail.jpg
Extract multiple thumbnails:
# 每 10 秒提取一帧
ffmpeg -i input.mp4 -vf fps=1/10 thumb%04d.jpg
# 提取前 10 帧
ffmpeg -i input.mp4 -vframes 10 frame%04d.png
8. Compression and Optimization
Reduce file size while maintaining quality.
Video compression (balanced mode):
ffmpeg -i input.mp4 -c:v libx264 -crf 23 -preset medium -c:a aac -b:a 128k output.mp4
High compression (smaller file):
ffmpeg -i input.mp4 -c:v libx264 -crf 28 -preset veryslow -c:a aac -b:a 96k output.mp4
Web-optimized compression:
ffmpeg -i input.mp4 -c:v libx264 -preset medium -crf 23 -movflags +faststart -c:a aac -b:a 128k output.mp4
Platform-specific presets
YouTube optimization
ffmpeg -i input.mp4 \
-c:v libx264 -preset slow -crf 18 \
-c:a aac -b:a 192k \
-pix_fmt yuv420p \
-movflags +faststart \
youtube.mp4
Instagram Stories (9:16)
ffmpeg -i input.mp4 \
-vf "scale=1080:1920:force_original_aspect_ratio=decrease,pad=1080:1920:(ow-iw)/2:(oh-ih)/2" \
-c:v libx264 -preset medium -crf 23 \
-c:a aac -b:a 128k \
-t 15 \
instagram_story.mp4
Twitter/X (16:9, maximum 2 minutes 20 seconds)
ffmpeg -i input.mp4 \
-vf scale=1280:720 \
-c:v libx264 -preset medium -crf 23 \
-c:a aac -b:a 128k \
-t 140 \
twitter.mp4
TikTok(9:16)
ffmpeg -i input.mp4 \
-vf "scale=1080:1920:force_original_aspect_ratio=decrease,pad=1080:1920:(ow-iw)/2:(oh-ih)/2" \
-c:v libx264 -preset medium -crf 23 \
-c:a aac -b:a 128k \
-t 60 \
tiktok.mp4
Common use cases
Screen recording optimization
# 减小屏幕录制文件大小
ffmpeg -i screen_recording.mov \
-c:v libx264 -preset medium -crf 23 \
-vf "scale=1920:-1" \
-c:a aac -b:a 128k \
optimized.mp4
Batch conversion
# 将所有 MOV 文件转换为 MP4
for i in *.mov; do
ffmpeg -i "$i" -c:v libx264 -crf 23 -c:a aac "${i%.mov}.mp4"
done
Create a video from images
# 从图片序列创建
ffmpeg -framerate 30 -pattern_type glob -i '*.jpg' \
-c:v libx264 -pix_fmt yuv420p \
output.mp4
# 单张图片转视频(5 秒)
ffmpeg -loop 1 -i image.jpg -c:v libx264 -t 5 -pix_fmt yuv420p output.mp4
Best practices
-
Always check the input file first:
ffmpeg -i input.mp4 # 或使用 ffprobe 获取详细信息 ffprobe -v quiet -print_format json -show_format -show_streams input.mp4 -
Use
-c copyto avoid re-encoding whenever possible:ffmpeg -i input.mp4 -ss 00:01:00 -t 30 -c copy output.mp4 -
Before processing, use
-tparameters to preview the result:# 先测试前 10 秒 ffmpeg -i input.mp4 -t 10 [其他参数] test.mp4 -
Use appropriate CRF values:
- 18 = visually lossless
- 23 = high quality (default value)
- 28 = acceptable quality, smaller file
- Range: 0 (lossless) to 51 (worst quality)
-
Add
-movflags +faststart:- to Web videos to enable progressive playback
- Move metadata to the beginning of the file
Error handling
When using this skill, always:
- Verify that the input file exists and is readable
- Check whether ffmpeg is installed before processing
- Verify that the output path is writable
- Handle errors gracefully with appropriate prompts
- Show progress when processing large files
Claude Usage Guide
When the user requests audio/video processing:
- Identify the task type
- Select the appropriate command from this skill
- Verify prerequisites(ffmpeg installed, input file exists)
- Explain what the command does before executing it
- Execute the commandand perform error handling
- Verify the outputwas created successfully
- Provide optimization suggestions(if applicable)
For complex workflows, execute step by step and explain each step.
Video concatenation notes: Use printf with process substitution whenever possible to avoid temporary files (see Method 2 in the concatenation section). Use a temporary list.txt file only when necessary.
Example
User:"Convert this MOV file to MP4" Response: MOV to MP4 command using H.264 encoding
User:"Make a GIF from this video, starting at the 10th second and lasting only 5 seconds" Response: GIF creation command with a time range
User:"I need to scale this 4K video down to 1080p for the web" Response: Combine resolution scaling and the Web optimization preset
User:"Extract audio as MP3" Response: Audio extraction command using MP3 encoding
References
- FFmpeg official documentation:https://ffmpeg.org/documentation.html
- FFmpeg Wiki:https://trac.ffmpeg.org/wiki
- Supported codecs:https://ffmpeg.org/ffmpeg-codecs.html
- Filter documentation:https://ffmpeg.org/ffmpeg-filters.html
Text extraction and transcription features. When it comes to transcription and text extraction features, refer to the following documentation:
Voice Model API (local speech transcription)After the application starts, it automatically runs on http://127.0.0.1:18923 starts a local HTTP interface for external callers such as Claude Skill to use.
API list
1. Query model status
GET /v1/audio/models/status
Response example:
{
"installed": true,
"modelDir": "/Users/xxx/.newmax/models/sensevoice-small",
"modelSize": 239549735
}
{
"installed": false,
"modelDir": "/Users/xxx/.newmax/models/sensevoice-small",
"missingFiles": ["model.int8.onnx", "tokens.txt"]
}
2. Audio transcription
POST /v1/audio/transcriptions
Content-Type: application/json
Request parameters:
| Field | Type | Required | Description |
|---|---|---|---|
| audioPath | string | Yes | Local absolute path of the audio file |
| language | string | No | Language code (e.g., zh, en, ja), leave blank for automatic detection |
Request example:
{
"audioPath": "/Users/xxx/Downloads/interview.mp3",
"language": "zh"
}
Success response (200):
{
"success": true,
"text": "第一段转录文字\n第二段转录文字\n第三段转录文字"
}
Failure response (400/500):
{
"success": false,
"error": "模型未安装,请先下载模型"
}
Usage instructions
- The port is fixed to
18923, and it only listens on127.0.0.1(local access) - Supported formats: WAV / MP3 / FLAC / M4A / OGG / AAC (non-WAV formats require ffmpeg to be installed on the system)
- Long audio is automatically transcribed in 30-second segments, and the results are concatenated with newline characters
- The model must first be downloaded in Settings → Voice Model (about 239 MB), otherwise the transcription API will return a not-installed error
- Uses the SenseVoice Small model, supporting 50+ languages including Chinese, English, Japanese, Korean, and Cantonese
Quick curl test
# 检查模型是否已安装
curl http://127.0.0.1:18923/v1/audio/models/status
# 转录音频文件
curl -X POST http://127.0.0.1:18923/v1/audio/transcriptions \
-H 'Content-Type: application/json' \
-d '{"audioPath": "/path/to/audio.mp3"}'
Calling in Claude Skill
# 在 skill 脚本中调用转录 API
result=$(curl -s -X POST http://127.0.0.1:18923/v1/audio/transcriptions \
-H 'Content-Type: application/json' \
-d "{\"audioPath\": \"$AUDIO_FILE\"}")
echo "$result" | jq -r '.text'