SuperAPI AI Directory
Partner中文AI newsFavorites
Skillsaudio-gen

audio-gen

Agents & models

View repository

Updated 10-15 23:23

About this skill

audio-gen

Overview

An integrated skill for AI-based audio generation. Using ElevenLabs' high-performance TTS engine, it generates natural voices in 32 languages (including Korean), and clones personal/brand voices from just a 1-minute sample. It also supports multilingual video dubbing (automatic lip sync), sound effect generation, and building real-time conversational AI voice agents.

Key Features

  • TTS (Text-to-Speech): Natural voice generation in 32 languages (optimized for Korean)
  • Voice Cloning: Clone personal/brand voices from a 1-minute sample
  • Multilingual Dubbing: Korean video → English/Japanese, etc. (automatic lip sync adjustment)
  • Sound Effect Generation: Sound effects for movies, games, and content
  • ConvAI: Real-time conversational voice agent (chatbot, AI agent)

Trigger Keywords

Use this skill for the following requests:

  • Voice generation related: "make me a voice", "TTS", "speech synthesis", "narration recording", "AI voice actor"
  • Voice Cloning: "clone my voice", "create a brand voice", "voice cloning", "learn from a sample voice"
  • Dubbing related: "English dubbing", "Japanese translation + recording", "foreign language subtitles + voice", "multilingual version"
  • Sound Effects: "generate sound effects", "sound effect", "background sound", "video sound"
  • Conversational AI: "AI agent", "voicebot", "real-time voice conversation", "automated phone answering"

Workflow

1. Basic TTS (Text-to-Speech)

1. 텍스트 입력 (한국/영어/32개국어)
2. 보이스 프리셋 선택 또는 커스텀 보이스 ID 지정
3. 모델 선택 (eleven_multilingual_v2 기본)
4. 오디오 생성 (MP3/WAV)

2. Voice Cloning

1. 참조 오디오 업로드 (1분 이상, 무음 최소화)
2. 보이스 이름/설명 입력
3. 클로닝 실행 → 보이스 ID 발급
4. TTS에서 클로닝된 보이스 사용

3. Multilingual Dubbing

1. 원본 비디오 업로드
2. 원본 언어 감지 (예: 한국어)
3. 타겟 언어 선택 (예: 영어, 일본어, 스페인어)
4. 자동 번역 + 보이스 생성 + 립싱크 매칭
5. 더빙된 비디오 다운로드

4. Sound Effect Generation

1. 효과음 설명 프롬프트 작성 (예: "천둥소리, 폭풍우")
2. 지속시간 설정 (1~30초)
3. 생성 및 다운로드

Usage Examples

Example 1: Generate Korean narration

"이 블로그 글을 한국어 나레이션으로 읽어줘.
여성 차분한 톤으로, 3분 분량."
→ 보이스: Rachel (여성 차분)
→ 모델: eleven_multilingual_v2
→ 출력: MP3 파일

Example 2: Brand voice cloning

"우리 CEO의 1분 연설 음원이 있어.
이 목소리를 클로닝해서 신제품 발표 나레이션을 만들어줘."
→ 참조: ceo_sample.wav
→ 출력: 클로닝된 보이스 ID + 나레이션 MP3

Example 3: English dubbing for a YouTube video

"이 한국어 교육 영상을 영어와 일본어로 더빙해줘.
원본 자막은 유지하고, 립싱크도 맞춰줘."
→ 입력: korean_tutorial.mp4
→ 출력: english_dub.mp4, japanese_dub.mp4

Example 4: Sound effect generation

"판타지 게임용 마법 시전 효과음 3개 만들어줘.
1. 화염구 (2초)
2. 얼음 폭발 (3초)
3. 치유 빛 (2.5초)"
→ 출력: fireball.wav, ice_explosion.wav, heal_light.wav

Output Format

TTS Output

  • File Format: MP3 (default), WAV, FLAC
  • Sample Rate: 44.1kHz, 48kHz
  • Channels: Mono/stereo selectable
  • Maximum Length: Unlimited (character count limited depending on your plan)

Voice Cloning

  • Voice ID: voices/xxxxx format
  • License: Cloned voices are permanently retained within the account
  • Sharing: Can be reused in other projects via the voice ID

Dubbing Output

  • Video Format: MP4 (original quality preserved)
  • Audio Track: Add multilingual audio tracks
  • Subtitles: Automatically generate SRT subtitle files
  • Sync: Automatic lip-sync adjustment (±100ms precision)

Notes

API key required

ELEVENLABS_API_KEY environment variable is required.

  1. elevenlabs.io Sign up
  2. Settings → API Keys → Create API Key
  3. .env or register it as a system environment variable: 
    export ELEVENLABS_API_KEY="your_api_key_here"

Pricing

Plan Price Characters Use case
Free $0 10,000자/월 Testing, personal projects
Starter $5/월 30,000자/월 Small-scale content
Creator $22/월 100,000자/월 YouTubers, freelancers
Pro $99/월 500,000자/월 App integration, commercial use

Limitations

  • Voice cloning: Requires at least 1 minute of sample (minimize silent segments)
  • Dubbing: Videos over 10 minutes require separate consultation
  • Commercial use: Requires Pro plan or higher (check license terms)
  • Rate limits: Free plan has RPM (requests per minute) limits

Model selection guide

Model Character Purpose Notes
eleven_multilingual_v2 Highest quality Branding, advertising, narration Optimized for Korean, recommended
eleven_flash_v2_5 Ultra-low latency Real-time conversation, gaming Under 200ms
eleven_turbo_v2_5 Cost-effective Long-form narration, bulk generation 50% cheaper

Default (ElevenLabs prebuilt) voice presets — Korean synthesis possible with eleven_multilingual_v2

Code Character 톤 Use case
Rachel Female calm Narrator Documentary, news, education
Antoni Male friendly Conversational Interviews, podcasts
Bella Female lively Energetic Advertising, promotional videos
Callum Male neutral Professional/narrator Reports, presentations

Related skills

  • higgsfield-video: Combine TTS audio with AI video
  • higgsfield-image: Generate images for audio content such as album art and thumbnails
  • moai-content:blog: Convert blog posts → narration scripts
  • moai-content:copywriting: Ad copy → ad voice generation

MCP server setup

This skill uses ElevenLabs MCP (stdio).

{
  "elevenlabs": {
    "command": "/bin/bash",
    "args": ["-l", "-c", "exec uvx elevenlabs-mcp"],
    "env": {
      "ELEVENLABS_API_KEY": "${ELEVENLABS_API_KEY}"
    }
  }
}

MCP server registration procedure: see moai-media/CONNECTORS.md .