Recreate pixel-perfect motion design from any reference video — by MEASURING it, not eyeballing it. Then flip the same system to render something entirely your own.
Every camera move, shape, colour and cut in the final prompt is a calculated value extracted with computer vision. You are never prompting blind.
Requirements
- Python 3.10+ with
opencv-python-headlessandnumpy(pip install opencv-python-headless numpy— use a venv if your system blocks pip) - The Higgsfield MCP connected to Claude Code (
claude mcp add --transport http --scope user higgsfield https://mcp.higgsfield.ai/mcp)
The pipeline (run it in order)
Stage 1 — Measure the reference
python scripts/measure.py reference.mp4 -o spec.json
Computer vision reads every frame and pulls out:
- Structure: cuts scored as frame-diff ratio against the local median, fitted black-window and white-flash detectors → the beat list
- Camera: Farneback optical flow on lit pixels, solved per frame pair for a similarity transform (scale / rotation / translation) with physical admissibility guards → push-in, pull-back, pan, roll, per-frame values
- Elements: connected-component blobs at each beat's midframe → shape class, normalized position and size
- Palette: k-means over lit pixels per beat → locked hex values
- Look: background floor, grain, bloom notes
spec.json is the contract. Later stages read it and never re-derive a number from pixels.
Stage 2 — Convert the spec into a structured prompt
python scripts/spec_to_prompt.py spec.json -o prompt.json --style "neon motion graphics on black"
Builds a shot-by-shot JSON video prompt: one shot per scene beat, transitions carried from the measured black/flash windows, camera phrased from the measured motion, palette locked to the measured hexes.
Stage 3 — Render through the Higgsfield MCP
Inside Claude Code, with the Higgsfield MCP connected, ask:
"Read prompt.json and generate this video on Higgsfield. Use the Google Omni model if it is available in the marketplace, otherwise Seedance 2.0. 9:16, 1080p, honor every shot duration, camera move and palette exactly."
The agent lists the available Higgsfield models, picks the conversational JSON-prompt model, fires the generation, polls the job, and returns the video URL.
Stage 4 — Your own sauce (the point of the system)
Copying proves the control. Now feed the system YOUR direction instead of a reference:
- Edit
spec.jsondirectly — change palettes, swap camera moves, retime beats - Or keep a folder of reference clips (your inspiration bank), measure each one, and mix beats across specs into one prompt
- Add your own custom shapes, fonts and camera moves by describing them in the shot descriptions
Then re-run stage 2 + 3. The render follows your numbers with the same precision it followed the reference's.
Rules for the agent
- NEVER skip the measurement stage and improvise a prompt — the measured values are the whole point.
- NEVER re-derive numbers from pixels after
spec.jsonis written; edit the spec instead. - If the Higgsfield MCP is not connected, stop and give the user the one-line install command above.
- Verify the model actually available on Higgsfield before claiming it (list models first; prefer Google Omni for JSON prompts, fall back to Seedance 2.0).
- Aspect ratio 9:16 and 1080p by default unless the user says otherwise.