create-music-elevenlabs

create-music-elevenlabs

Purpose

Generate original music for ads and short-form videos using the ElevenLabs Music API. Three modes:

• Quick prompt — one-shot text-to-music for a bed.
• Plan-then-compose — generate a structured composition plan (sections, styles, lyrics), edit it, then compose. Use when you need control over arrangement, instrumental vs vocal sections, or specific lyrics.
• Video-to-music — upload one or more video clips and get a score matched to the cut.

Pairs with create-programmatic-music-bed for VO-aware mix/ducking guidance and with create-voiceover-elevenlabs for full ad audio.

Inputs

• Brief or prompt describing the desired music.
• Target duration in milliseconds (3000–600000). Required for prompt mode.
• Optional composition_plan.json for structured control.
• Optional source video(s) for video-to-music (≤10 videos, ≤200MB combined, ≤600s total).
• Output directory for artifacts.
• ELEVENLABS_API_KEY in env — OR FAL_API_KEY for the Fal-hosted fallback (see "Fal AI fallback" below).

Mode selection

Need	Use	Endpoint
Quick bed	scripts/compose.sh	POST /v1/music
Fine control over sections / lyrics	scripts/plan.sh then scripts/compose_from_plan.sh	POST /v1/music/plan, POST /v1/music
Word timestamps for caption sync	scripts/compose_detailed.sh --with-timestamps	POST /v1/music/detailed
Score matched to a video edit	scripts/video_to_music.sh	POST /v1/music/video-to-music
Progressive playback / preview	scripts/stream.sh	POST /v1/music/stream

The composition-plan endpoint is free (no credit cost), so prefer plan-then-compose when iterating on structure.

Fal AI fallback (when ELEVENLABS_API_KEY is unavailable)

Fal AI hosts ElevenLabs Music behind the fal-ai/elevenlabs/music endpoint and accepts FAL_API_KEY for auth. Useful when:

• The project's .env lacks ELEVENLABS_API_KEY but has FAL_API_KEY (common — Fal is a popular multi-model proxy).
• ElevenLabs direct API is rate-limited or down.
• The team's ElevenLabs tier doesn't include music (FAL aggregates without tier checks).

Same underlying music_v1 model. Cost: ~$0.40 per 28s track.

Submit + poll pattern (Bash + curl):

FAL_API_KEY="$(grep '^FAL_API_KEY' /path/to/.env | cut -d= -f2)"

# Submit
RID=$(curl -sS -X POST https://queue.fal.run/fal-ai/elevenlabs/music \
  -H "Authorization: Key $FAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"prompt": "<music brief>", "music_length_ms": 28000}' \
  | python3 -c 'import json,sys; print(json.load(sys.stdin)["request_id"])')

# Poll until done
until curl -sS "https://queue.fal.run/fal-ai/elevenlabs/requests/$RID/status" \
        -H "Authorization: Key $FAL_API_KEY" \
      | grep -qE '"status":"(COMPLETED|FAILED)"'; do
  sleep 4
done

# Fetch result
URL=$(curl -sS "https://queue.fal.run/fal-ai/elevenlabs/requests/$RID" \
        -H "Authorization: Key $FAL_API_KEY" \
      | python3 -c 'import json,sys; print(json.load(sys.stdin)["audio"]["url"])')
curl -sSL "$URL" -o audio/music.mp3

Constraints (same as direct API): music_length_ms ∈ 3000–600000. The Fal wrapper accepts only prompt + music_length_ms in the simple call — no composition_plan parameter exposed. For structured drops, generate two tracks separately and splice with FFmpeg (see LEARNINGS.md #18).

Workflow

1. Read the brief; pick a mode from the table above.
2. Run the matching script. Save artifacts under the requested output directory.
3. Verify duration with ffprobe -v error -show_entries format=duration -of csv=p=0 music.mp3.
4. Write manifest.json with status, mode, prompt or plan path, output paths, model, seed, output_format, and any warnings.
5. Write a short verification.md listing checks performed.

Key parameters

• prompt (string) — text description. Mutually exclusive with composition_plan.
• composition_plan (object) — MusicPrompt with positive_global_styles, negative_global_styles, and sections[] (each with section_name, positive_local_styles, negative_local_styles, duration_ms, lyrics).
• music_length_ms (int) — 3000–600000. Per-section: 3000–120000.
• model_id — defaults to music_v1.
• force_instrumental (bool) — default false. Set true for VO ads.
• respect_sections_durations (bool) — default true.
• seed (int) — for reproducibility (best-effort).
• output_format (query) — defaults to mp3_44100_128. Higher bitrates require Creator+ tier.
• with_timestamps (bool, detailed only) — returns word-level timing.
• sign_with_c2pa (bool) — C2PA-sign the MP3.

Output

• music.mp3 (or chosen codec) — primary audio.
• composition_plan.json — if planned mode.
• music_metadata.json — when using compose_detailed.sh (includes timestamps when requested).
• manifest.json — status, mode, inputs, outputs, model, seed, warnings, errors.
• verification.md — duration check, instrumental check, mix notes.

Quality checks

• music.mp3 exists and is decodable by FFmpeg.
• Measured duration is within ±1s of music_length_ms (or sum of section durations for plan mode).
• If force_instrumental: true, audio contains no vocals.
• For VO videos, follow create-programmatic-music-bed mix guidance: lower bed level and apply sidechain ducking under VO during assembly.
• License/usage note recorded (ElevenLabs music — verify your tier's commercial-use terms).

Failure modes

• Missing ELEVENLABS_API_KEY — fall back to Fal endpoint if FAL_API_KEY is set.
• 422 validation error: both prompt and composition_plan set, or section duration outside 3000–120000ms, or total length outside 3000–600000ms.
• 403: bitrate / store_for_inpainting not allowed on current tier.
• Video-to-music: > 10 videos, > 200MB combined, or > 600s combined.
• Generated audio contains vocals when an instrumental bed was required (set force_instrumental: true).
• Output duration drifts from request when respect_sections_durations: false.
• Single-prompt structural drop doesn't render. Even with explicit "Section A → Section B at " prompting, the model produces one continuous song. For real structural drops, commission two separate tracks (e.g. Phase A + Phase B) and butt-splice via FFmpeg concat. The 50-100ms boundary silence works AS the drop. See LEARNINGS.md #18 and skills/orchestrators/higgsfield/edit-video.md § Phase 3a.
• FAL's fal-ai/elevenlabs/sound-effects endpoint is upstream-broken (as of recent runs). Submitted jobs reach COMPLETED status but the result-fetch returns HTTP 400 with Sound effect generation failed: ... invalid_model_id: eleven_text_to_sound_v0. Expected one of: ['eleven_text_to_sound_v2']. Passing model_id: eleven_text_to_sound_v2 in the payload is silently ignored. The music endpoint (fal-ai/elevenlabs/music) is unaffected. For SFX, use the synthesize-sfx-ffmpeg atom instead — it produces production-quality bell dings, bag thuds, punch impacts, whooshes, risers, and breath SFX from scratch via ffmpeg sine waves + decay envelopes, with no API dependency. See LEARNINGS.md #24.