split-audio-by-words

Purpose

Cutting a long-form podcast clip into per-scene mp3s with raw timestamps always lands on a mid-word boundary sometimes. The klarify scene-07 split clipped at source-49.2s but the word "stakes" started at 49.4s — the per-scene file ended at "...higher" and the rest of "stakes situations" was lost. We didn't notice for two iterations.

This atom solves it the right way: transcribe the source with word-level timestamps, then snap every requested cut point to the nearest word boundary (with operator-tunable preference for snapping before/after the boundary word).

Inputs

<audio> — required. The long-form source mp3/m4a/wav.
<scenes> — required. Either:
- explicit: a JSON file or --scene flag with [{ "scene": 1, "caption": "...", "start": 15.1, "end": 18.1 }, ...]
- auto-pause: pass --auto-split and an optional --min-pause-duration 0.6 (seconds). The atom uses ffmpeg silencedetect + Whisper word timing to identify sentence-ending pauses and emit one split per sentence.
<provider> — groq (default — cheapest, fastest), fal, or openai. Composes atoms/source/transcribe-audio-fal for the fal path; uses inline Groq/OpenAI clients otherwise.
<snap_mode> — nearest (default), before, or after. Controls which word boundary to snap to when a requested cut lands inside a word.
<padding> — default 0.20s. Appended to each scene's end with afade=t=out:st=...:d=0.20 so the cut tail fades smoothly instead of clipping abruptly.
<output_dir> — default <audio_parent>/voiceovers/. Per-scene files written as scene-NN-<speaker?>.mp3.

Workflow

Step 1 — Transcribe with word timestamps

If <provider>=fal, compose atoms/source/transcribe-audio-fal (passing chunk_level=word). Otherwise call Groq Whisper or OpenAI Whisper inline with timestamp_granularities=[word]. Output: words.json with [{word, start, end}, ...].

This cache is reused for subsequent runs on the same source — re- transcribing wastes credits.

Step 2 — Snap cut points to word boundaries

For each requested (start, end) pair:

def snap(t, words, mode):
    """Snap timestamp t to nearest word boundary based on mode."""
    # Find word w such that w.start <= t < w.end
    inside = next((w for w in words if w["start"] <= t < w["end"]), None)
    if not inside:
        # t is in a silence — already on a clean boundary, no snap needed
        return t
    if mode == "before":
        return inside["start"]
    if mode == "after":
        return inside["end"]
    # nearest
    return inside["start"] if (t - inside["start"]) < (inside["end"] - t) else inside["end"]

Write snapped timestamps to the manifest with a snapped_from field so the operator can see what changed.

Step 3 — Extract per-scene mp3s

ffmpeg -y -ss "$snapped_start" -to "$snapped_end" \
  -i "$audio" \
  -af "afade=t=out:st=$((snapped_end - snapped_start - padding)):d=$padding" \
  "$output_dir/scene-$NN-$speaker.mp3"

The afade=t=out masks any tail noise and provides a clean audio-cut boundary downstream (especially important when the audio gets muxed under B-roll where there's no visual cue covering an abrupt cut).

Step 4 — Write manifest.json

{
  "source": "audio/source-practice-stakes-low-51s.mp3",
  "provider": "groq",
  "scenes": [
    {
      "scene": 1,
      "speaker": "TAMMER",
      "caption": "you don't start",
      "text": "You don't start practicing layups",
      "clip_start": 15.10,
      "clip_end": 18.06,
      "snapped_from": { "start": 15.10, "end": 18.10 },
      "duration": 2.96,
      "file": "scene-01-tammer.mp3"
    },
    ...
  ]
}

Step 5 — Verification report

For each scene, run word-level transcription on the extracted file (small, cheap) and verify:

First word starts within the first 200ms
Last word ends at least 200ms before the file end (i.e. the afade caught the tail)
Transcribed text matches the manifest's text field within Levenshtein 0.9

Surface failures to stderr; operator decides whether to re-snap or accept.

Output

<output_dir>/scene-NN-<speaker>.mp3 per scene
<output_dir>/manifest.json
<output_dir>/words.json (full source transcript — reusable for burn-in-captions later)
<output_dir>/verification.md (per-scene transcribed-text check)

Quality checks

Every scene's actual file duration matches clip_end - clip_start within ±20ms
No two scenes overlap in source timestamps
Each scene's afade tail is present (last ~200ms of audio amplitude decays smoothly to silence)
Verification step confirms transcribed text aligns with manifest

When to use

Splitting a podcast clip into per-scene VO files for lipsync (klarify pipeline)
Re-cutting an existing master into scene-aligned chunks for B-roll experiments
Inside molecules/podcast-clip-animated-ad (step 2)

Do NOT use for:

General audio editing where word-boundary precision isn't needed (use raw ffmpeg -ss/-to)
Splitting music tracks (this atom is speech-focused; for music use atoms/audio-editing/)

Failure modes

Whisper misses a word — the snapped timestamp may be off by one word. Verification step catches this. Operator can pass explicit timestamps to override.
Long source with many speakers — Whisper's word stream blends them; pass --speaker-diarization to use a smarter provider (currently unsupported — flagged for future work).
Auto-split detects too many micro-pauses — increase --min-pause-duration from 0.6s to 1.0s or 1.5s.

Defaults rationale

Knob	Default	Why
provider	groq	$0.001/min — cheapest, fastest
snap_mode	nearest	Minimizes timestamp drift from operator intent
padding	0.20s	Long enough for an inaudible afade-out, short enough not to clip the next scene's first word
min-pause-duration	0.6s	Catches sentence boundaries while ignoring breath pauses

Example

python3 .../skills/atoms/source/split-audio-by-words/scripts/split.py \
  --audio audio/source-practice-stakes-low-51s.mp3 \
  --scenes-json scenes.json \
  --output-dir voiceovers/ \
  --provider groq \
  --padding 0.2

Where scenes.json is:

[
  {"scene": 1, "speaker": "TAMMER", "start": 15.10, "end": 18.10, "caption": "you don't start", "text": "You don't start practicing layups"},
  {"scene": 2, "speaker": "TAMMER", "start": 18.10, "end": 23.60, "caption": "when the game starts", "text": "and basketball, when the game starts because the pressure is a little bit higher."}
]

Output:

voiceovers/scene-01-tammer.mp3 (2.96s, snapped from operator's 3.0)
voiceovers/scene-02-tammer.mp3 (5.50s)
voiceovers/manifest.json
voiceovers/words.json
voiceovers/verification.md

Implementation note

Script at scripts/split.py. Composes atoms/source/transcribe-audio-fal when provider=fal; otherwise calls Groq/OpenAI directly. Caches the full-source words.json in the output dir so a re-run with different scene boundaries doesn't re-transcribe.

The companion atom atoms/review/review-transcript-integrity should be run on the final master to catch any per-scene splits that ended up truncating despite the snap (rare but possible if Whisper missed the boundary word).