SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillssound-design

sound-design

Design & creation

View repository

Updated 10-15 23:47

About this skill

Sound Design

SFX + music beds for video edits, via the ElevenLabs API. The hierarchy never changes: the voiceover is dominant, SFX punctuate, the bed is felt, not heard. Every number below came from shipped edits that survived client review — treat them as defaults, not suggestions.

Requirements

  • ELEVENLABS_API_KEY — export it, or put ELEVENLABS_API_KEY=… in a .env at the repo root. Both scripts read the environment first, then walk up from cwd looking for a .env. They error clearly if neither is set. Never hardcode the key.
  • ffmpeg + ffprobe on PATH (or FFMPEG_PATH / FFPROBE_PATH env vars).
  • Node >= 20 (the scripts use global fetch).
  • python3 (stdlib only) for the optional BPM estimator in references/style-matching.md.

1. SFX kit — scripts/gen-sfx.mjs

Endpoint: POST https://api.elevenlabs.io/v1/sound-generation with body {"text": prompt, "duration_seconds": d, "prompt_influence": 0.6} and header xi-api-key. duration_seconds minimum is 0.5 — the API 400s below it (the script clamps).

The one non-negotiable: audit every file. Roughly 1/3 of generations come back near-silent or quiet. A kit that skips the audit ships hits nobody hears. The script automates the whole loop:

node .claude/skills/sound-design/scripts/gen-sfx.mjs kit.json sfx/

kit.json is a list of {name, duration, prompt} (see assets/kit.example.json for a proven starter kit):

{
  "normalize_db": -3,
  "min_peak_db": -12,
  "effects": [
    { "name": "stamp-slam", "duration": 0.8,
      "prompt": "loud heavy thud slam impact, stamp hitting a desk, punchy bass" }
  ]
}

Per effect it: generates → measures the peak with ffmpeg volumedetect → any file peaking below min_peak_db is a dud, regenerated once with "loud, punchy: " prepended → then the whole kit is peak-normalized to −3dB (quiet survivors get auto-boosted, hot ones trimmed) so composition volume values mean the same thing for every file. It also flags files whose first sound starts later than ~0.15s (late onsets land off-beat — regenerate those with "sharp attack" in the prompt or trim the head) and writes durations.json for the composition builder.

Prompt-writing tips that held up:

  • Name the physical source + character: "stamp hitting a desk, punchy bass", not "impact sound". Add "single" for one-shots, "dry, no music" to stop the model inventing a score behind the effect.
  • Whooshes are a taste risk. A reviewer on a shipped edit killed every whoosh/swoosh as "cheesy". Default transitions to silence or a soft pop; reserve whooshes for section cuts (see mixing rules below).
  • Match sound semantics to on-screen motion: a counter draining to zero gets a power-down / falling pitch, never a rising tick. A rising sound on a "losing" beat gets flagged in review.

2. Music bed — scripts/gen-music.mjs

Endpoint: POST https://api.elevenlabs.io/v1/music with body {"prompt": "...", "music_length_ms": N}.

node .claude/skills/sound-design/scripts/gen-music.mjs \
  --prompt "..." --seconds 32 --out music/bed.mp3

The script pre-checks the prompt, requests a couple of padded seconds by default, trims to length with a fade-out, and reports duration + peak.

ToS trap — artist names in prompts → HTTP 400 (film and brand names are likely filtered the same way). "In the style of " fails; the 400 body often includes a usable prompt_suggestion, which the script surfaces. Describe the STYLE in words instead. A bed prompt should cover, in one sentence each:

  • genre ("dark cinematic electronic underscore")
  • bpm ("around 120 bpm")
  • drum character ("tight punchy kick, quiet ticking hi-hats")
  • bass ("deep sustained sub bass")
  • melody policy ("minimal melodic content, no lead melody")
  • energy arc ("steady energy with a gentle build in the final third")
  • mix intent — always include verbatim: "mixed quiet and dry to sit under a spoken voiceover"
  • "no vocals"
  • "seamless loop feel"

To match a reference track the user hands you ("make it feel like this"), do NOT try to name or describe the song — profile it and prompt by descriptors only. Full workflow, including the spectrogram read and a ~30-line BPM estimator, in references/style-matching.md.

Generated tracks often open with a 2–4s low-energy warm-up. If the bed must slap from frame one (paired with a sub-drop open), scan per-second volumedetect means over the first ~10s, find where energy plateaus, and cut the bed with -ss <onset> plus a 0.25s fade-in.

3. Sub-drops / risers as openers

The single highest-impact opener: a short sub-drop under the first frame. Generate it through the SFX endpoint (add it to kit.json):

{ "name": "sub-drop", "duration": 1.2,
  "prompt": "massive cinematic sub bass drop, falling pitch, punchy, no music" }

Verify the sub is actually there — laptop speakers can't hear it, and the model sometimes returns a mid-range "boom" with no real low end:

# low-band energy: max here should be within ~6dB of the file's full-band max.
# 20dB+ down = no real sub — regenerate.
ffmpeg -i sfx/sub-drop.mp3 -af "lowpass=f=120,volumedetect" -f null -

# 0–800Hz spectrogram: a real drop is a dense low block with a falling-pitch trail;
# a dud is empty below ~120Hz. Compare against a known-good drop side by side.
ffmpeg -i sfx/sub-drop.mp3 -lavfi "showspectrumpic=s=1024x400:stop=800" drop-spec.png

After the final render, re-check the opener window the same way (-ss 0 -t 1.4 + lowpass=f=120,volumedetect on the master).

4. Mixing into the composition

Order of operations matters — every ratio below assumes step 1 happened first.

  1. Loudness-normalize the VO first, before setting any bed or SFX level:

    ffmpeg -i vo.wav -af loudnorm=I=-14:TP=-1.2:LRA=11 -ar 48000 vo-norm.wav

    Mixing against an un-normalized VO makes every number below wrong.

  2. Music bed: track volume ~0.13–0.18 (0–1 linear gain — HyperFrames data-volume, or the track-volume slider in any NLE). Against a loudnorm'd VO and a bed peaking near 0dB that puts the bed ~19–20dB under VO peaks. Start at 0.15; offer 0.18 as the "louder" option. In a word gap the bed should raise the gap's mean level by a few dB versus a no-music render — present, not competing.

  3. SFX: per-hit base volume × one kit master gain, capped. Keep per-hit taste values in the ~0.28–0.5 range by type, then scale the whole kit with a single master gain (~1.95 proved right for a −3dB-normalized kit against a loudnorm'd VO), capping the product at 0.8 so slams never bury the voice:

    Hit type base × 1.95, cap 0.8
    soft whoosh 0.28–0.35 0.55–0.68
    pop / tick 0.32 0.62
    count-up / cha-ching 0.40–0.45 0.78–0.80
    stamp / impact / sub-drop 0.50 0.80 (capped)

    The one-number kit gain is the point: "everything a touch louder/quieter" becomes a single edit instead of forty.

  4. Placement rules (each of these was learned in review):

    • Land every hit on the exact word ("fire" SFX on the word fire), not at the sentence start.
    • Whooshes on section cuts only — a whoosh on every card reads cheesy.
    • One biggest swell per video (usually the opener sub-drop or the final reveal). If everything is big, nothing is.

5. Verify the mix — in dB, never by ear alone

Every audio bug ever caught in this pipeline was found by measuring, not listening.

# window around a hit (hit at t=12.4, file 0.8s): start ~0.05s early
ffmpeg -ss 12.35 -t 0.9 -i final.mp4 -af volumedetect -f null -

# a quiet window — a word gap with no hits and no big VO peak
ffmpeg -ss 8.0 -t 0.8 -i final.mp4 -af volumedetect -f null -

# integrated loudness of the master
ffmpeg -i final.mp4 -af ebur128 -f null -    # read the final "I:" line

Expectations:

  • A hit window whose peak matches the VO's peaks is audible-subtle; +3–5dB over is punchy. A hit window that measures the same as the quiet window means the hit is missing or buried — fix placement or volume.
  • The quiet window should sit a few dB above the same window in a no-music render (the bed is present) but far below VO peaks (the bed isn't competing).
  • Integrated loudness target: ~−14 to −13 LUFS. Much below −14, the whole mix is quiet for social feeds; above ~−12, the limiter is doing the mixing for you.

Files