audio-to-srt

audio-to-srt: audio → clean SRT

When to trigger

The user provides an audio/video file (mp3/wav/m4a/mp4/mov/mkv) and requests:

• Convert to subtitles, generate SRT, add subtitles
• Speech to text with timecodes
• Subtitle cleanup, typo correction, sentence-break polishing

Two routes

Route	Model	Timecode granularity	Speed	Privacy	Use case
A. Groq (default)	whisper-large-v3-turbo	word-level	Fast	Cloud	General scenarios, need speed and accuracy
B. Local Whisper	medium	segment-level	Slow	Fully local	Sensitive content, no network

Only via Groq can you use resegment.py for precise sentence breaking. Local Whisper can only preserve the original segments.

Core principles (most important, violation means failure)

1. Timecodes are sacred and inviolable: during text cleanup, each SRT block's HH:MM:SS,mmm --> HH:MM:SS,mmm linemust not be changed at all
2. Paragraph boundaries must not be moved: during the cleanup stage, do not merge two segments, do not split a segment, do not add or delete segments (only the resegment stage may split, and it must be based on word-level timecodes)
3. Only change text, not meaning: fix typos, add punctuation, smooth the phrasing, but do not add or remove content

---

Route A: Groq workflow (default)

Step 1: Environment check

# Groq API Key（擇一）
echo $GROQ_API_KEY                # 環境變數
ls ~/.groq_api_key                # 或本地 key 檔

If neither is present → prompt the user to set it up, and abort.

ffmpeg check (used for downsampling large files):

ffmpeg -version 2>&1 | head -1

Step 2: Prepare working directory

Create _subtitles/ alongside the audio file to store intermediate artifacts.

Step 3: Automatic compression for large files (built in)

Groq's limit is 25 MB; exceeding it returns 502 Bad Gateway(blocked by Cloudflare). transcribe_groq.py When a file > 24 MB is detected,automaticallyuse ffmpeg to compress it to a 16kHz / mono / 32kbps temporary file for upload, then delete it after completion—seamless for the user. A 78-minute video was measured going from 75 MB → 18 MB, and recognition quality remained good.

If it is still > 24 MB after compression (rare), the script will abort and prompt. In that case, split it manually.

Step 4: Groq STT, produce word-level JSON

python "%USERPROFILE%/.claude/skills/audio-to-srt/scripts/transcribe_groq.py" \
  "輸入檔.mp3" \
  --out _subtitles/輸入檔.groq.json

Produce verbose_json, including segments and words(each word has start/end).

Step 5: Re-segment based on word-level timecodes → raw SRT

python ".../scripts/resegment.py" \
  _subtitles/輸入檔.groq.json \
  --audio "輸入檔.mp3" \
  --out _subtitles/輸入檔.raw.srt

Split-point priority: strong punctuation 。！？ > segment boundary > weak punctuation ，、 > hard split (search backward for punctuation). Parameters:MAX_DUR=3.0s、MAX_CHARS=15、MIN_DUR=0.6s。

Prevent orphaned trailing fragments (2026-07 fix): the old version would cut "the last character of a sentence" to the beginning of the next segment (e.g., the previous segment ends with '…the program you need' and the next segment begins with 'gram, completely free!'). Two layers of protection are now built in:

1. Lookahead guard punct_ahead(): segment boundary splitting (Rule 2) and hard over-limit splitting (Rule 5) Before cutting, first check whether there is a punctuation ending within the next 1–2 words; if so, be lenient and do not cut, and wait for the punctuation to end it naturally.
2. Orphan rescue rescue_orphan_tails(): after cutting, do a final scan; whenever the previous segment does not end with punctuation and the segment starts with a leftover tail of "≤2 characters + punctuation", move the whole batch back to the previous segment. Discourse markers ("好," "嗯," "欸," ...) are whitelist-protected at both layers and will not be misjudged or mistakenly moved. Measured results: for two 40-minute videos, true orphan leftover tails went from 11→0 and 16→0, and segments beginning with discourse markers remained intact.

--audio will, after sentence splitting, call ffmpeg silencedetect to detect true silence (default -35 dB, 0.25 s), shrink each segment's end to the silence start and extend the next segment's start to the silence end, so subtitles automatically leave blank space at silences. When --audio is not passed, the old behavior is maintained (Groq word timestamps themselves have no silence gaps).

Step 6: Apply vocabulary corrections (mechanical replacement)

python ".../scripts/apply_vocab.py" \
  _subtitles/輸入檔.raw.srt \
  --out _subtitles/輸入檔.vocab.srt

The script contains a REPLACEMENTS list:

• GPT-Codex variant(including DexDex/Dex Dex/dex dex → Codex): must be handled first to avoid misjudgment after Cloud→Claude
• Claude ecosystem: ClockCode/CloudCode/ClawCode → Claude Code, 克勞德 → Claude
• Cloud → Claude(place last to avoid affecting Cloud Code first)
• NotebookLM、GPT-Image 2 and other AI tools
• Common typos: 斷考→段考, 三十八→三師爸, 小課→小克, etc.

Only edit text lines; never touch timecodes.

Step 7: Claude paragraph-by-paragraph refinement

• Use Read to read 輸入檔.vocab.srt
• according to references/cleanup_rules.md clean paragraph by paragraph (fix punctuation, delete redundant words)
• For each paragraph, only modify the text after line 3
• Produce _subtitles/輸入檔.clean.srt

Text-cleaning checklist:

• Is the number of segments equal to vocab.srt? (must not increase or decrease)
• Have the timecode lines not been changed at all?
• references/vocabulary.md Is the vocabulary presented correctly?

Step 8: Verification

python ".../scripts/validate_srt.py" \
  --raw _subtitles/輸入檔.vocab.srt \
  --clean _subtitles/輸入檔.clean.srt

The number of segments is consistent, timecodes match one by one, are monotonically increasing, and the text is non-empty.

Step 9: Produce plain text transcript

python ".../scripts/srt_to_txt.py" \
  _subtitles/輸入檔.clean.srt \
  --out 輸入檔.txt

Suitable for: YouTube descriptions, IG/FB posts, cover quotes, post-production scripts.

Step 10: Delivery

Place two files in the same directory as the audio：

• 輸入檔.srt — subtitle file (copied from clean.srt)
• 輸入檔.txt — plain text transcript

_subtitles/ Keep intermediate files for auditing. Report: total segments, total duration, average segment length, txt character count, main modification types.

---

Route B: Local Whisper workflow (fallback)

Use this route when the user explicitly requests local processing, or when there is no Groq Key.

PYTHONIOENCODING=utf-8 PYTHONUTF8=1 python -X utf8 -m whisper "輸入檔.mp3" \
  --model medium \
  --language zh \
  --output_format srt \
  --output_dir ./_subtitles \
  --initial_prompt "以下為繁體中文。專有名詞：Claude、Claude Code、NotebookLM、Gemini、Groq、Whisper、Obsidian、三師爸、小克。"

Windows pitfalls：

1. whisper CLI is not in PATH → use python -m whisper
2. cp950 cannot write Traditional Chinese → must add PYTHONUTF8=1 and -X utf8

After output, skip Step 5 (resegment is not applicable; segment-level timecodes can only be kept as-is), and go directly to Step 6 (apply_vocab) → Step 7 (text cleaning) → Step 8 (verification) → Step 9 (convert to plain text) → Step 10 (delivery).

---

File structure

skills/audio-to-srt/
├── SKILL.md                      # 本檔
├── scripts/
│   ├── transcribe_groq.py        # Groq STT，產 word-level JSON
│   ├── resegment.py              # 依 word-level 時間碼重新斷句
│   ├── apply_vocab.py            # 詞彙機械替換（只動文字行）
│   ├── srt_to_txt.py             # SRT → 純文字稿
│   └── validate_srt.py           # 時間碼驗證
└── references/
    ├── cleanup_rules.md          # 清字規則（逐段不跨段）
    └── vocabulary.md             # 自訂詞彙表

Route selection decision tree

有 GROQ_API_KEY？
├─ 有 → 路線 A（Groq）
│       └─ 檔案 > 25 MB？→ 先 ffmpeg 降至 16k mono 再傳
└─ 無 → 提示使用者設 key；若使用者要求離線 → 路線 B（本地 Whisper）

Pitfalls encountered

• Whisper recognition is accurate but sentence splitting is poor → solved by Groq word-level + resegment
• Long audio runs out of memory (local) → switch to Groq; Groq 25 MB limit → ffmpeg downsampling
• Cloud / Claude / Codex misidentify each other → apply_vocab order matters: handle GPT-Codex variants first (including DexDex), then Claude, and finally Cloud→Claude
• Windows cp950 encoding → always add for the local route PYTHONUTF8=1
• Chinese filenames uploaded to Groq have broken encoding → inside transcribe_groq.py switch to audio.<ext> upload