SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillscreate-lipsync-veed-fal

create-lipsync-veed-fal

Coding & development

View repository

Updated 10-15 21:54

About this skill

create-lipsync-veed-fal

Purpose

The same Veed Fabric 1.0 model as create-lipsync-hedra, called through fal.ai's gateway instead of Hedra's. Use this when:

  • You already have a FAL_API_KEY set up and don't want a Hedra subscription.
  • You're doing a one-off batch and pay-per-call billing is cleaner than maintaining a Hedra credit balance.
  • Your Hedra account ran out of credits mid-batch and you want to finish the remaining clips on fal.

Same output quality, same lipsync model, same input contract. Only the gateway + billing differ.

Pricing (as of 2026-05)

  • 480p: $0.08 per second of generated video
  • 720p: $0.15 per second of generated video

A 3-second 720p clip = $0.45. A 23-clip ad with ~60s total audio = ~$9. Compare to Hedra Creator plan ($30/mo) — fal wins for one-off batches, Hedra wins for ongoing use.

Pricing source: fal.ai/models/veed/fabric-1.0.

Inputs

Required:

  • image_path — local path to character still (PNG or JPG, < 10 MB). Tight head-and-shoulders, neutral closed-mouth pose. Lipsync only works if the mouth starts in a neutral state — see the lessons in create-lipsync-hedra/SKILL.md for why.
  • audio_path — local path to dialogue audio (MP3, WAV, M4A, or AAC, < 10 MB).
  • output_path — local path where the resulting MP4 should be saved.

Optional:

  • resolution — "480p" (cheaper, slightly lower quality) or "720p" (default, ad-grade).

Credentials:

Preflight

test -n "$FAL_API_KEY" || test -n "$FAL_KEY" || { echo "Missing FAL_API_KEY / FAL_KEY in .env"; exit 1; }
python3 -c "import fal_client" || pip3 install fal_client
command -v ffprobe >/dev/null || { echo "ffprobe required"; exit 1; }

Workflow

The script scripts/generate.py handles upload → submit → poll → download.

python3 skills/atoms/lipsync/create-lipsync-veed-fal/scripts/generate.py \
  --image /path/to/character.png \
  --audio /path/to/line.mp3 \
  --output /path/to/clip.mp4 \
  --resolution 720p

What the script does:

  1. Upload image to fal storage — fal_client.upload_file(image_path) returns a public https://v3.fal.media/... URL.
  2. Upload audio to fal storage — same call, same return shape.
  3. Submit generation — fal_client.subscribe("veed/fabric-1.0", {image_url, audio_url, resolution}). This is the synchronous pattern; it blocks until the result is ready (~1.5 min at 480p for 10s of audio, ~5 min at 720p).
  4. Download the resulting video to output_path.
  5. Verify with ffprobe that duration matches input audio within ±10%.
  6. Write <output_path>.meta.json with {request_id, image_url, audio_url, duration_sec_video, duration_sec_audio, size_bytes, model: "veed/fabric-1.0", resolution} for cost tracking + debug.

The script uses fal_client.subscribe (sync) rather than the queue API because per-clip durations are 1-5 min — fine for one-shot calls. For very large batches (>50 clips) prefer fal_client.submit_async to parallelize without blocking threads.

Output

  • <output_path> — MP4 talking-head clip with audio baked in.
  • <output_path>.meta.json — request metadata + cost-tracking fields.

Quality Checks

  • Output file > 100 KB and exists.
  • ffprobe reports duration within ±10% of input audio.
  • Output codec is h264 + AAC.
  • meta.json includes request_id for retry / dispute.

Failure Modes

Symptom Likely cause Fix
401 Unauthorized Bad / missing FAL_API_KEY Verify env var; key starts with fal- or just a long hex string.
429 Too Many Requests Account rate limit fal has per-account RPS limits. Drop concurrency to 2-3.
Generation takes >10 min fal queue is slow (peak hours) Patient; or switch to fal's async submit API and poll yourself.
Lipsync drift mid-clip Audio file has leading silence Trim with ffmpeg -ss 0.1 before passing in.
Body waves / hands move unnaturally Input still has loose clothing / wide framing Crop tighter, head-and-shoulders only. Veed Fabric freezes the bust but not loose hands.
Mouth doesn't move Audio asset upload failed silently Check meta.json for valid audio_url and that it returns the audio bytes when fetched.

Limitations

  1. Bust framing only. Veed Fabric animates head/face. The body is mostly frozen — body motion below shoulders is unstable.
  2. 10 MB max per file. Both image and audio must be under 10 MB. For long audio (>3-4 minutes), split into multiple clips.
  3. Resolution caps at 720p. For higher resolution, upscale in post (e.g. RealESRGAN) or use a different model.
  4. No text_prompt parameter. Unlike Hedra, fal's veed/fabric-1.0 doesn't accept scene description — the model is pure image+audio.
  5. Two resolution options only. 480p ($0.08/s) and 720p ($0.15/s). No 540p or 1080p.

When to use this vs create-lipsync-hedra

Use case Recommended
One-off ad, no existing Hedra account veed-fal
Recurring ad production, weekly batches hedra (Creator plan amortizes well)
Hedra account out of credits mid-batch veed-fal for the remaining clips
Need exact same look as previous batch (continuity) Stick with whichever you started — outputs are model-identical but minor sampling variance exists between gateways
Need text_prompt for scene direction (rare for ad lipsync) hedra (supports text_prompt)

References