SuperAPI AI Directory
Partner中文AI newsFavorites
Skillsvideo-use

video-use

Agents & models

View repository

Updated 10-29 04:36

About this skill

Video Use — Conversational Video Editing with Coding Agents

video-use enables natural language conversational video editing through coding agents. Instead of dumping thousands of raw video frames into the model context, it reads a compact word-level transcript (takes_packed.md, ~12KB) derived from ElevenLabs Scribe, and drills into visuals on demand via timeline_view (filmstrip + waveform + word labels PNG) only at decision points.

All user footage remains untouched. All outputs are written to <videos_dir>/edit/.

When to use this skill

  • Editing raw multi-take footage through natural conversation with Claude Code or Codex.
  • Cutting out filler words (umm, uh, false starts) and dead space between takes.
  • Auto color grading footage segments (warm cinematic, neutral punch, or custom FFmpeg chains).
  • Applying clean audio cuts with 30ms audio fades at segment boundaries to prevent pops.
  • Generating and burning custom styled subtitles (2-word uppercase chunks, karaoke, or lower thirds).
  • Adding animated overlays generated via HyperFrames, Remotion, Manim, or PIL in parallel sub-agents.
  • Persisting multi-session editing context and decisions in <videos_dir>/edit/project.md.

When NOT to use this skill

  • Converting long-form recordings (podcasts, webinars) into vertical 9:16 highlight clips with speaker tracking → use autoclip.
  • Generating short-form videos from scratch using a topic or keyword with stock media and TTS → use moneyprinter-turbo.
  • Building code-first React video compositions with Remotion from scratch → use video-production.
  • Creating product launch promo videos using 2.5D page camera recipe cards → use video-shotcraft.
  • Driving a native Rust nonlinear video editor GUI or MCP server → use filmcraft or palmier-pro.

Instructions

Step 1: Verify environment and prerequisites

  1. Check that ffmpeg and ffprobe are installed and available on PATH.
  2. Ensure ELEVENLABS_API_KEY is set in the environment or in .env at the video-use repository root.
  3. Install dependencies via uv sync or pip install -e . inside the video-use workspace.
  4. Verify user footage exists in a target folder: 
    ls -la /path/to/raw/videos

Step 2: Ingest and transcribe footage

  1. Run parallel batch transcription on the directory: 
    python helpers/transcribe_batch.py /path/to/raw/videos
  2. Pack raw transcript JSONs into phrase-level markdown: 
    python helpers/pack_transcripts.py --edit-dir /path/to/raw/videos/edit
  3. Read takes_packed.md (~12KB) to inspect verbal takes, speaker turns, and silence gaps.

Step 3: Converse and propose strategy

  1. Discuss the material with the user: target duration, target aspect ratio (16:9 vs 9:16), pacing feel, must-keep moments, and visual aesthetic.
  2. Formulate a 4–8 sentence editing strategy covering take choices, cut plan, subtitle style, and overlay slots.
  3. Wait for user confirmation before generating cut lists or rendering.

Step 4: Execute edit and hard correctness rules

Produce edl.json and render clips adhering strictly to production correctness rules:

  • Rule 1 (Subtitles Last): Subtitles are burned LAST in the filter chain after all overlays.
  • Rule 2 (Lossless Concat): Per-segment extract with grading → lossless -c copy concat.
  • Rule 3 (Audio Fades): Add 30ms audio fades at every segment boundary (afade=t=in:st=0:d=0.03,afade=t=out:st={dur-0.03}:d=0.03).
  • Rule 4 (Overlay PTS): Shift overlay start via setpts=PTS-STARTPTS+T/TB.
  • Rule 5 (Master SRT): Align subtitle timestamps to output timeline offsets (output_time = word.start - segment_start + segment_offset).
  • Rule 6 (Word-boundary Snap): Snap every cut point to a word boundary; never cut inside a spoken word.
  • Rule 7 (Cut Padding): Apply 30–200ms padding around cut edges to absorb ASR drift.

Render draft preview:

python helpers/render.py /path/to/raw/videos/edit/edl.json -o /path/to/raw/videos/edit/preview.mp4 --preview --build-subtitles

Step 5: Self-evaluate rendered output

Before presenting the final video:

  1. Run timeline_view.py on the rendered output at cut boundaries (±1.5s window).
  2. Inspect for visual jump cuts, waveform spikes (audio pops), or subtitle masking.
  3. Verify integrated audio loudness via FFmpeg ebur128.
  4. Fix any defect and re-render (capped at 3 self-eval iterations).

Examples

Example 1: Full-take talking head cleanup

User prompt:

Here is my 3-take YouTube tutorial footage. Cut out all stumbles and pauses, make it concise, add clean yellow subtitles, and export final.mp4.

Workflow:

  1. Run transcribe_batch.py on the footage folder.
  2. Read takes_packed.md, select the best take for each section, and list cut boundaries at silences ≥ 0.4s.
  3. Present the 5-bullet editing plan to user.
  4. On approval, generate edl.json with 30ms audio fades and word-snapped cuts.
  5. Render edit/final.mp4 with styled subtitles and confirm audio/visual continuity.

Best practices

  • Never frame-dump: Never feed thousands of video frames into LLM context. Rely on takes_packed.md and call timeline_view.py only at ambiguous pauses or retake forks.
  • Keep source footage immutable: Never overwrite original camera files. All edits, EDL files, and intermediate clips belong in <videos_dir>/edit/.
  • Preserve emotional punchlines: Do not trim reactions or laughs immediately after a speaker finishes a sentence. The reaction is part of the beat.
  • Persistent project memory: Append session summaries, approved cut decisions, and user preferences to project.md so follow-up sessions maintain continuity.

References