remix-music-video-ad-from-sample

remix-music-video-ad-from-sample

The generative counterpart to the chat-reveal remix wrappers (iMessage / Apple Notes / ChatGPT). Unlike those, this format is paid (~$15-25 of ElevenLabs + Higgsfield + Kling gen per run) and long (~30 min wall-clock), so the orchestration shape is the same but the creative phases drive a generative pipeline: original anthem with word timestamps → 14 beat-locked tableaux → keyframes → image-to-video → cinematic captions. The reference look the design comes from is Hype and Vice "Game Day Girls" — 35mm Kodak Portra, a game-day anthem, 14 tableaux on a tailgate arc. Two human gates exist (storyboard approval BEFORE any paid gen, then the T01 aesthetic test before the batch of 13); this wrapper folds both into Phases 1.5 / 2 and never starts paid gen before storyboard approval.

Purpose

Given one published cinematic music-video ad sample (from the Goose Ads library) + a researched brand, produce a finished cinematic music-video ad for that brand. The source sample defines the shape — the 14-tableau 3-act arc (anticipation → energy → reflection), the structure- locked roles (INTRO → … → END_CARD), the cut-on-vocal-beat principle, the one-look-pack-per-run rule, the music vibe/bpm — and this skill swaps the substance: the tableaux re-frame the brand's product/garment as the object-hero, the look pack is chosen to fit the brand, the anthem lyrics are rewritten in the brand's campaign voice, and the END_CARD tableau carries the brand's real wordmark + tagline.

This skill is orchestration: it derives + gets approval for the brand's concept.json, then hands rendering to create-cinematic-music-video and handles all Gooseworks I/O (render rows, stage reporting, durable upload, final pin). Creative craft rules (the 10 states S0–S9, the look-pack STYLE_OPENER/NEGATIVE_TAIL injection, the beat-anchored TIMELINE, the captions preset, the baked-in workarounds) live in the create molecule — do not duplicate or override them here.

It runs on the user's machine inside their Claude Code session (the goose-video local-worker model). The runtime contract — MCP tools, proxies, durable URLs — is the installed ads-remix master skill; this recipe assumes it. All paid gen routes through the Gooseworks proxies so it bills the user's credits.

Inputs

Input	Required	Notes
project_id	yes (app flow)	A draft Gooseworks ad project with source_sample_id + format_key: "music-video". The paste-prompt carries it. Without one, create_ad_project { brand_id, name, source_sample_id, format_key: "music-video" } first.
source sample	from project	get_ad_template { template_id: <source_sample_id or slug> } → recipe.concept (the source concept.json: brand, run_slug, title, campaign, duration_s, look_pack, palette_anchors, music (vibe/bpm/structure_note), lyrics_locked_md, accent_words, and the 14 tableaux[] each with id/role/lyric_anchor/prompt/motion_hint), extracted_script (the source anthem lyrics), how_to, media_url (watch it if the arc/cut feel is unclear).
brand	yes	Researched brand pack (research_status: "complete"); else run the brand-research recipe first. Needs: voice/tone (for lyric register), brand palette, product/garment imagery (for object-hero framing), wordmark/logo (for the END_CARD), tagline / campaign line.
angle	optional	User-supplied steer ("lean nightlife so use CineStill", "campaign line: every day is game day", "make the anthem about the morning rush"). Folded into the concept draft (look pack, lyric theme, accent words).
machine	yes	ffmpeg with libass (the renderer uses the imageio-ffmpeg static binary — Homebrew ffmpeg lacks libass on macOS), the Higgsfield + ElevenLabs proxies authenticated, and the renderer molecule + its atoms resolving (preflight below).

Composed Atoms

• create-cinematic-music-video — the renderer (sibling molecule): the full S0–S9 state machine — storyboard, ElevenLabs anthem with word timestamps, Higgsfield keyframes (T01 gate then batch), Kling image-to-video, stitch, loudnorm, libass caption burn. This skill never re-implements its 10 states. (skills/molecules/music-video/create-cinematic-music-video)
• atoms/music/create-music-elevenlabs — the original anthem with word-level timestamps (renderer state S2); routed through the Gooseworks ElevenLabs proxy so it bills the user's credits.
• atoms/image-generation/create-image-gpt-image-fal — the 14 tableau keyframes via Higgsfield gpt_image_2 (T01 aesthetic-test gate, then the batch of 13); routed through the Higgsfield / FAL proxy.
• atoms/video-generation/create-video-kling — image-to-video on each keyframe (14 clips, kling image-to-video); routed through the FAL proxy.
• atoms/captions/burn-in-captions — cinematic lower-third caption burn with the libass-enabled static binary (renderer state S9).
• atoms/review/watch — watch the rendered master MP4 (frames + transcript) for the output QC pass.

Workflow

Phase 0 — Fetch + gates (no render row yet)

1. get_ad_project { project_id } → keep source_sample_id, app_url, brand_url, brand_id.
2. get_ad_template { template_id: source_sample_id } → keep recipe.concept, extracted_script, how_to, media_url. If recipe.concept is missing, reconstruct the arc from extracted_script (the locked lyrics map to the lyric anchors of the 14 structure-locked roles) and /watch the media_url to confirm the cut feel and look pack. Never invent a structure the source doesn't have.
3. Brand gate: get_ad_brand { brand_id }.
   • research_status: "complete" → REUSE the existing pack — do NOT re-run research. Read it over MCP: agent-config/brands/<slug>/brand-research/*.md (voice, palette, never-say) + brand-assets/manifest.json (product/garment imagery, wordmark). Re-research is a ~10-minute spend the user already paid for; repeating it on a complete brand is a bug, not thoroughness.
   • Not complete → run the master skill's "Set up a brand" workflow (idempotent), then continue. For failed specifically: the pack on disk is usually fine (the last run crashed before finalize) — inventory it first and patch only the gap (often assets + manifest + finalize_brand_research, ~30s, zero research).
   • Only exception: the status says complete but the pack files are missing/unreadable, or the user explicitly asked to refresh the research — say which case applies before re-running.
4. Machine preflight (gen is expensive — fail fast and friendly, never open a row that can only burn credits and fail):
   • ffmpeg -version and confirm libass is available — the renderer's caption burn (S9) shells out to the imageio-ffmpeg static binary, because Homebrew ffmpeg lacks libass on macOS. Confirm pip show imageio-ffmpeg (or python3 -c "import imageio_ffmpeg"). A Homebrew-only ffmpeg fails the caption stage with No such filter: 'subtitles'.
   • Higgsfield + ElevenLabs proxy access authenticated — the anthem (S2) and the keyframes (S4/S5) call out over the Gooseworks proxies; confirm the proxy token resolves (npx goose-video@latest login if not). Without it every paid stage 401s.
   • The renderer molecule + its atoms resolve — create-cinematic-music-video's driver (scripts/one_shot.py) and its composed atoms (create-music-elevenlabs, create-image-gpt-image-fal, create-video-kling, burn-in-captions) are present on the machine.
   • If any is missing, tell the user exactly what to install (pip install imageio-ffmpeg, npx goose-video@latest login) or to run goose-video doctor, and STOP — do not open a render row, and do not spend a cent of paid gen.

Phase 1 — Derive the brand's concept.json

Map the source concept onto the brand. Keep the load-bearing structure, swap the substance.

• Keep the source's structure (this is why this ad works):
  • The 14-tableau 3-act arc — anticipation → energy → reflection — with the roles structure-locked: INTRO → PERSONAL_BEAT_1 → PERSONAL_BEAT_2 → WIDE_ENSEMBLE → KINETIC_CLOSE_1 → KINETIC_CLOSE_2 → SUSPENDED_TIME → HOOK_HERO → KINETIC_PEAK → PRODUCT_HERO → MICRO_MONTAGE → ORIGIN_WINK → TWILIGHT_OUTRO → END_CARD. Don't skip slots — even a 20s cut compresses each tableau, it does not remove it. The 3-act arc is what separates this from generic AI-music-video slop.
  • The cut-on-vocal-beat principle — an original anthem with word-level timestamps so every visual cut lands on a vocal beat (the create molecule's S2→S3 gives ~80–90 word anchors).
  • The one-look-pack-per-run rule — a single photographic style across all 14 tableaux; the through-line is what makes the spot read as one ad. Same brand mark on every tableau.
• Swap the substance into the brand:
  • Object-hero framing of the brand's product/garment — "Hero subject: a [brand product] on a [surface]" — with people only in supporting roles (the create molecule defaults to object-hero to dodge gpt_image_2 NSFW false-flags; keep that discipline). Pull the product/garment from brand-assets/manifest.json.
  • The brand's palette_anchors (3–4 colors from the brand-research palette).
  • A look_pack chosen to fit the brand: KODAK_PORTRA_DAY (warm afternoon / golden hour / lifestyle), CINESTILL_800T_NIGHT (neon-noir / nightlife / winter), PAPER_CUT_CRAFT (Y2K / candy-coded / CPG / wellness), or — if none fits — a new pack added per the create molecule's "Look packs" section (a one-file PR; this wrapper does not author it). The look pack is the aesthetic swap point.
  • New anthem lyrics in the brand's campaign voice — written into source/lyrics-locked.md and mirrored into the lyrics_locked_md field; accent_words updated to the brand's terms (name, campaign words). The lyrics carry the brand's emotional arc, never feature callouts.
  • The brand's real wordmark + tagline on the END_CARD tableau (role 14) — the wordmark is the brand's actual mark, the tagline its real campaign line.
• Compliance: zero source-brand leakage — no source brand name, SKU, school, campaign line, or competitor reference may survive anywhere in the lyrics, any tableau prompt/motion_hint, the accent_words, or the end card.

Phase 1.5 — Storyboard approval gate (in-session)

The storyboard is the in-session approval surface — and the renderer's Gate 2, the cheapest review surface, which must clear before any paid gen starts. Show the user the drafted concept as a readable storyboard: the look pack + music vibe/bpm, the anthem lyrics, and the 14 tableaux (each id · role · lyric_anchor · a one-line description of prompt). Ask them to approve or edit. Persist the draft so the app can see it: write_file → agent-config/brands/<slug>/projects/<project_id>/working/concept.json, append_project_message { role: "agent", content: <the storyboard script> }, and mirror it to the project row so the APP's Script panel shows it: update_ad_project_script { project_id, script_drafts: { format: "music-video", look_pack, concept }, script: <the readable look-pack + lyrics + 14-tableau storyboard lines> }. Do not start paid gen until the user says go — a render here is real money (~$15-25) and ~30 min of their machine. If they edit, update concept.json + re-mirror and re-confirm once.

Draft-only mode (free review): if the user's instruction says "draft only" / "storyboard only" / "just the concept", STOP right after the mirror above — no render row, no paid gen, zero credits spent. Tell the user the storyboard is now visible on the project page in the app, and that re-running the prompt (or replying "render it") completes the ad.

No-review escape: if the user's instruction says to skip review ("no review", "single-shot", "don't ask for approval"), skip the pause entirely — still write working/concept.json + the append_project_message script mirror, then render immediately. In this mode auto-approve both renderer gates (Gate 2 storyboard and Gate 3 T01) so the run completes without stopping.

Phase 2 — Render (open the row FIRST, report stages)

Renders are long and expensive (~$15-25 of paid gen), so unlike the static flow the render row opens BEFORE generating — and paid gen must not start before storyboard approval (the row opening is bookkeeping, not the spend). The renderer's Gate 3 (T01 aesthetic test) lives inside the keyframes stage: generate T01 first, look at it (auto-approve it under the no-review escape), then commit to the batch of 13.

1. submit_render { project_id, kind: "full" } → keep render_id.

2. update_render_status { render_id, status: "running", stage: "storyboard" } — then keep the stage current as you work, mapped to the renderer states. The app shows it live; a silent >10 min run shows as stalled:

   stage	renderer states	when
   storyboard	S0–S1	concept approved, storyboard built (Gate 2 cleared)
   music	S2–S3	ElevenLabs anthem generated + word timestamps adapted ms→s
   keyframes	S4–S5	Higgsfield gpt_image_2 — T01 first (Gate 3), then the batch of 13
   clips	S6	Kling image-to-video on each of the 14 keyframes
   stitch	S7	concat the 14 segments to the silent master
   captions	S8–S9	mix music + loudnorm to -14 LUFS, then libass caption burn
   export	—	final 1080×1920 9:16 master encode + thumbnail

3. Follow create-cinematic-music-video end-to-end with the approved concept.json. Local translations:

   • Scaffold the renderer's run layout in a temp working dir (NOT a brand repo folder): concept.json; source/lyrics-locked.md; audio/ (music.mp3 + music_metadata.json + words.json); assets/keyframes/T01..T14.png + assets/clips/T01..T14.mp4; working/segments/ + working/logs/; clips/ (master-silent.mp4); finals/master-final.mp4 + .ass. The driver scripts/one_shot.py --concept <…> walks the state machine and is idempotent (file-existence checks); resume after each gate with --continue.
   • Route all paid gen through the Gooseworks proxies so it bills the user's credits: the anthem (S2) via the ElevenLabs proxy, the keyframes (S4/S5) via the Higgsfield / FAL proxy, the image-to-video (S6) via the FAL proxy — same <apiBase>/api/internal/<vendor>-proxy + ?token= pattern the master skill installs. Never call a vendor API directly with a raw key.
   • Captions (S9) need the libass static binary — the create molecule auto-resolves the imageio-ffmpeg path; do not substitute Homebrew ffmpeg (it lacks libass and the burn fails).

Local render contract. Rendering is the create molecule's scripts/one_shot.py state machine driving its composed atoms (create-music-elevenlabs, create-image-gpt-image-fal, create-video-kling, burn-in-captions) — those atoms must be present on the machine. They ship with the installed ads-remix master skill (the goose-video local-worker installs them alongside this recipe), not inside the recipe folder. This wrapper never re-implements the 10 states; it only opens the render row, keeps the stage table current, routes gen through the proxies, and publishes. Two contracts the renderer enforces and this wrapper must respect: the TIMELINE is computed per-run from audio/music_metadata.json (never hard-code timings across runs), and the look pack is locked for all 14 tableaux (don't mix packs).

Phase 3 — QC by watching

Run atoms/review/watch on finals/master-final.mp4 (or step its frames + audio directly): the 14 tableaux play in the structure-locked role order, cuts land on the anthem's vocal beats, the look pack reads consistently across every tableau, the product is the brand's object-hero (not the source's), the END_CARD shows the brand's real wordmark + tagline + hero SKU, captions are the cinematic lower-third preset with no No such filter: 'subtitles' artifact, the master is 1080×1920 30fps h264+aac at -14 LUFS, and duration is within ±20% of the source's duration_s. Fix-and-re-render at most once per issue — re-run only the failed state via its script (e.g. one keyframe, one clip), not the whole pipeline; then run the create molecule's own Quality Checks list.

Phase 4 — Publish (durable URLs, then the links)

1. get_upload_url → PUT the master MP4 to agent-config/brands/<slug>/projects/<project_id>/working/final.mp4 (content_type: video/mp4); grab a poster frame (ffmpeg -ss 1 -frames:v 1, e.g. from the HOOK_HERO tableau) and PUT it as working/final-thumb.jpg.
2. update_render_status { render_id, status: "complete", output_url: "/api/ads/projects/<project_id>/render-file?path=working/final.mp4", thumbnail_url: "/api/ads/projects/<project_id>/render-file?path=working/final-thumb.jpg", stage: "export", duration_sec: <ffprobe seconds> } — the render-file path is the DURABLE form; never store a CDN or raw S3 URL.
3. One render → done (latest shows by default). Multiple kept versions → set_final_render with the best render_id.
4. End with BOTH links exactly as the master skill requires: app_url + brand_url, verbatim.

Decision Rules

• This skill vs create-cinematic-music-video: remixing a library sample for a brand → this skill (it feeds the create molecule). A net-new cinematic music-video from a brief with no source sample → the create molecule directly.
• Concept derivation source: recipe.concept when present (normal); else extracted_script + watching media_url. Never invent an arc the source doesn't have — the 14 roles are structure-locked.
• Look pack = the aesthetic swap point. There is no render-variant axis for this format (unlike iMessage's phone-frame/plain) — the axis is the LOOK PACK. Pick the pack that fits the brand (KODAK_PORTRA_DAY / CINESTILL_800T_NIGHT / PAPER_CUT_CRAFT, or a new pack added per the create molecule); keep one pack locked across all 14 tableaux. Ignore any recipe.render_variant on a music-video sample — it does not apply here.
• Gates → paid gen ordering. Storyboard approval (Gate 2) is mandatory before ANY paid gen starts. The T01 aesthetic test (Gate 3) sits inside the keyframes stage before the batch of 13. Honor the "no review" escape by auto-approving BOTH gates — never let it block a single-shot run, but never let paid gen start before the storyboard is at least persisted + mirrored.
• Approval: always pause at Phase 1.5 on the first render of a project. On an explicit re-run where the user already gave concept edits in the same message, apply them and proceed without a second pause. No-review escape: "no review" / "single-shot" / "don't ask for approval" → skip the pause, still write working/concept.json + the append_project_message mirror, auto-approve both renderer gates, then render immediately.
• Music is always original. This format generates the anthem (ElevenLabs Music) — never licensed music. Strip artist names from the music vibe (ElevenLabs ToS filter returns bad_prompt on them); use descriptive language ("indie-electronic anthem with female vocal grit"), not name references.
• Batch runs (the prompt lists several project ids): run Phases 0–1.5 for each first (collect all storyboard approvals in one message), then render sequentially — these runs are credit-heavy and the Higgsfield/Kling/ElevenLabs jobs and ffmpeg encodes thrash one machine; never fan out N expensive pipelines in parallel.

Output

• working/concept.json — the approved brand concept.json (in the project's Gooseworks folder), carrying the brand's look pack, palette anchors, anthem lyrics, accent words, and the 14 structure-locked tableaux.
• working/final.mp4 (1080×1920 9:16 master) + working/final-thumb.jpg in the project folder.
• A complete render row carrying the render-file output_url, thumbnail_url, duration_sec, and final stage; set_final_render pin when >1 version.
• A closing message with the project app_url + brand brand_url.

Quality Checks

• The 14 tableaux play in the structure-locked role order (INTRO → … → END_CARD); no slot skipped.
• Cuts land on the anthem's vocal beats (original anthem with word timestamps drives the TIMELINE).
• One look pack reads consistently across every tableau — the through-line makes it one ad.
• The product is the brand's object-hero in the relevant tableaux; people only in supporting roles.
• Anthem lyrics are in the brand's campaign voice (emotional arc, not feature callouts); accent words are the brand's terms.
• END_CARD shows the brand's real wordmark + tagline + hero SKU.
• Zero source-brand leakage anywhere (lyrics, tableau prompts/motion hints, accent words, end card) — compliance-critical.
• finals/master-final.mp4 is 1080×1920, 30fps, h264+aac, -14 LUFS, captions burned cleanly (no No such filter: 'subtitles'), duration within ±20% of the source's duration_s.
• duration_sec reported matches ffprobe; render row carries render-file URLs, not CDN links.

Failure Modes

• ffmpeg lacks libass / imageio-ffmpeg missing → caption burn (S9) fails with No such filter: 'subtitles'. Caught in Phase 0 preflight; tell the user pip install imageio-ffmpeg (or goose-video doctor); no render row opened.
• Higgsfield / ElevenLabs proxy not authenticated → every paid stage 401s. Caught in Phase 0 preflight — tell the user npx goose-video@latest login and STOP before opening a row or spending a cent.
• Renderer molecule / atoms missing (create-cinematic-music-video driver or a composed atom not resolvable) → the generative pipeline isn't installed. Caught in Phase 0 preflight — point at the molecule + atoms installed by the ads-remix master skill before opening a render row.
• Sample has no recipe.concept and no usable extracted_script → tell the user this sample isn't remix-ready yet and stop; do not improvise an arc (the admin needs to fill the sample's remix payload).
• Brand pack lacks a product/garment image or a wordmark → ask the user for the product image and the official wordmark (or a URL) before the storyboard gate; object-hero tableaux can't be generic stock and the END_CARD can't be CSS-faked text.
• S2 music gen returns bad_prompt → an artist name slipped into the music vibe; strip it and use descriptive language (the API response includes a prompt_suggestion). Owned by the create molecule's workarounds; surface it, don't re-implement.
• A single keyframe or clip fails (Higgsfield 502 / Kling timeout) → re-run only that one via the renderer's per-job script; don't re-run the whole pipeline (each rerun re-spends credits).
• Render dies mid-run → one retry of the failed state; on second failure update_render_status { status: "failed", error_message } and report — never leave the row running.
• Upload fails → retry the presigned PUT once; persistent failure → mark the render failed with the local file path in error_message so nothing is lost.

Tests

See tests/ — structural smoke, a sample project/sample input pair (a realistic recipe.concept grounded in the renderer's schema), the expected artifact list, a manual end-to-end script, and the verifier checklist.

Skill Location & Related

• This skill: skills/molecules/ad-format/remix-music-video-ad-from-sample/
• Renderer: create-cinematic-music-video
• iMessage counterpart (chat-reveal sibling, same orchestration shape): remix-imessage-ad-from-sample
• Registry entry: formats.json → "music-video" (the app + CLI route here through it)