gitframes-render

gitframes-render

Setup

Import everything from the gitframes npm package. If the project doesn't have gitframes set up yet, follow the gitframes skill first (install, project scaffold, first render).

Overview

gitframes renders headlessly in Node.js on top of native WebGPU (Dawn) and mediabunny WebCodecs hardware encoders. It does not run a browser, Puppeteer, or Chromium.

Because video rendering is resource-intensive and native GPU pipelines run asynchronously in worker pools, gitframes workflows enforce a strict verification ladder: inspect inexpensive intermediate artifacts first before committing to a full video render.

The 5-Step Verification Ladder

Always proceed through the ladder in order. Catching a layout or timing bug at step 1 or 2 takes seconds; catching it after a 30-second 4K video render wastes minutes.

Step 1: Static Typecheck & Schema Validation

Before touching the GPU, verify TypeScript types and parse the composition against the strict runtime schema:

import { CompositorProgramSchema } from "gitframes";

// 1. Compile spec
const spec = comp.toSpec();

// 2. Validate against strict schema
const parsed = CompositorProgramSchema.safeParse(spec);
if (!parsed.success) {
  console.error("Schema validation failed:", parsed.error.format());
  process.exit(1);
}

Typecheck the project:

npx tsc --noEmit -p .

Step 2: Single Frame Inspection (`renderFrame`)

Render isolated keyframe points (e.g. entry, peak action, exit) directly to PNG files on disk to inspect alignment, typography, colors, and opacity:

import fs from "node:fs/promises";
import path from "node:path";
import { HeadlessMediaRenderer } from "gitframes";

const renderer = new HeadlessMediaRenderer();

// Render frame 45 (1.5s at 30fps)
const pngBuffer = await comp.renderFrame({
  frame: 45,
  renderer,
});

await fs.writeFile("output/frame-45.png", pngBuffer);

Step 3: Motion & Layout Stability Contact Sheets (`renderFrameGrid`)

To verify easing curves, cut points, kinetic sweeps, or text wrapping without encoding a full video, generate a sequential frame grid image:

import fs from "node:fs/promises";
import { renderFrameGrid } from "gitframes";

// Samples frames across the intro sweep (frames 0 to 60)
const gridPng = await comp.renderFrameGrid({
  frames: [0, 6, 12, 18, 24, 30, 36, 42, 48, 54, 60],
  columns: 4,
  cellWidth: 480, // Downscaled thumbnail size for rapid visual scanning
  showLabels: true, // frame number and timestamp under each cell
});

await fs.writeFile("output/intro-grid.png", gridPng);

Contact sheets immediately reveal:

Motion hitches and non-smooth easing deceleration.
Text clipping or wrapping jumps across keyframe boundaries.
Bad layer z-indexing or missing cut transitions.

Step 4: Programmatic Conformance Probes in Vitest

Write automated headless vitest tests that assert pixel colors, transparency, or layout bounding boxes:

import { describe, it, expect } from "vitest";
import { HeadlessMediaRenderer } from "gitframes";
import { buildFilm } from "../film.js";

describe("Hero Scene Conformance", () => {
  it("renders non-black title text on frame 30", async () => {
    const comp = await buildFilm();
    const renderer = new HeadlessMediaRenderer();
    const png = await comp.renderFrame({ frame: 30, renderer });

    expect(png.length).toBeGreaterThan(1000);
    // Use skia-canvas or pixel probes to assert color at target coordinates
  });
});

Step 4.5: Show the User Before Exporting (`preview`)

When a person is in the loop, let them watch the composition before you spend time on the final render. startPreview serves a localhost player with the soundtrack, a timeline with the waveform, and frame stepping. The page loads the composition's own code and renders it with WebGPU in the browser, so playback runs at or near full frame rate; the server only serves that code, the project's files, and the soundtrack mixed by the export engine. Point it at the module that builds the composition:

import { startPreview } from "gitframes";

const session = await startPreview(
  { entry: new URL("./film.ts", import.meta.url), export: "buildFilm" },
  { title: "Launch film" },
);
console.log(`Preview at ${session.url}`);
await session.closed; // keep the process alive while the user watches

The export is a function returning the Composition (it may be async), or a Composition itself. The browser calls it, so build-time Node code in it must stay simple: node:path, fileURLToPath(import.meta.url) and synchronous fs reads (JSON, text) work through shims; writes are skipped. Assets referenced by absolute path load from the server, limited to the repository that holds the entry (root overrides it).
Run it in the background. The preview serves until it is stopped. Start the script as a background command with the longest timeout your harness allows, tell the user the URL, and ask for feedback in the chat. Feedback comes from the user, not from the page.
One tab for the whole loop. The port is fixed per project directory. Running the preview again (after you've made changes) takes over from the one already running, and a tab left open reconnects and reloads into the new version by itself. Don't ask the user to reopen anything. When the user closes the tab, the preview stops by itself about 5 s later (a reload doesn't stop it). If the preview was stopped (tab closed, time limit, or you killed it), just start it again.
Show it where the user is. startPreview only serves the page and returns the URL. If your app has a built-in browser (Claude Code's browser pane, Codex's), open the URL there. Otherwise give the user the link, or pass open: true to open their default browser (skipped when an open tab reconnects on its own). The page needs WebGPU (current Chrome, Edge or Safari).
Audio is the same mix renderVideo muxes (audio: false skips it). Playback uses the soundtrack as its clock; when a frame takes long (a video seek, a heavy shot), the clock and the sound hold until the picture catches up.

Step 5: Full Video Export (`renderVideo`)

Once static frames and motion grids are verified, encode the final MP4/WebM video:

import path from "node:path";

const result = await comp.renderVideo({
  outputPath: "output/final-video.mp4",
  quality: "high", // "draft" | "medium" | "high" | "lossless"
  codec: "avc",    // "avc" (H.264) | "hevc" (H.265) | "vp9" | "av1"
  audioCodec: "aac",
  concurrency: 4,  // Parallel frame encoding threads
});

// Always invoke cleanup to release temporary frame cache and file handles
await result.cleanup?.();

Pass qa: true to have the engine check the output while it renders (see Built-in Video QA below) and return the findings as result.qa.

Production Render Script Pattern

Production projects structure their src/render.ts to support both quick frame probes and full exports from the CLI:

// src/render.ts
import fs from "node:fs/promises";
import path from "node:path";
import { HeadlessMediaRenderer, startPreview } from "gitframes";
import { buildFilm } from "./film.js";

const OUT = path.resolve(import.meta.dirname, "../output");
const [mode = "video", ...args] = process.argv.slice(2);

if (mode === "preview") {
  // The browser runs film.ts itself; no need to build the film here.
  const session = await startPreview({ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" });
  console.log(`Preview at ${session.url}`);
  await session.closed; // serves until its tab closes or the next preview replaces it
  process.exit(0);
}

const comp = await buildFilm();
await fs.mkdir(OUT, { recursive: true });

if (mode === "frames") {
  const renderer = new HeadlessMediaRenderer();
  await fs.mkdir(path.join(OUT, "frames"), { recursive: true });
  for (const f of args.map(Number)) {
    const png = await comp.renderFrame({ frame: f, renderer });
    const file = path.join(OUT, "frames", `f${String(f).padStart(4, "0")}.png`);
    await fs.writeFile(file, png);
    console.log(`Rendered frame: ${file}`);
  }
} else {
  const started = Date.now();
  const result = await comp.renderVideo({
    outputPath: path.join(OUT, "output.mp4"),
    quality: "high",
  });
  await result.cleanup?.();
  console.log(`Exported video: ${result.filePath} in ${((Date.now() - started) / 1000).toFixed(1)}s`);
}

// CRITICAL: Always explicitly exit process to prevent native WebGPU/mediabunny thread hangs
process.exit(0);

CLI Usage:

# Render specific keyframes for instant review:
tsx src/render.ts frames 0 18 36 72

# Live player for the user (run in the background; re-run after changes and the open tab reloads):
tsx src/render.ts preview

# Render full master video:
tsx src/render.ts

Built-in Video QA (`renderVideo({ qa })`)

The engine measures the video while it renders: every frame as it is read back for the encoder, and the audio as it is mixed. No second decode pass, and the numbers describe exactly what was encoded.

import { formatQaReport } from "gitframes";

const result = await comp.renderVideo({
  outputPath: "output/final-video.mp4",
  qa: { targetLufs: -14 }, // or `qa: true` for defaults (no loudness target)
});
console.log(formatQaReport(result.qa!));
if (!result.qa!.passed) process.exitCode = 1; // fail CI / the agent loop

result.qa is structured, so branch on it instead of parsing logs:

`issues[].code`	Severity	Meaning
`frames_missing`	error	Fewer frames encoded than the composition has
`black_frames`	warning	≥ `minBlackSec` (0.5 s) of black frames; has `startSec`/`endSec`
`audio_clipping`	warning	Samples at or beyond full scale
`loudness_off_target`	warning	Integrated loudness more than `lufsTolerance` (1 LU) from `targetLufs`
`animation_truncated`	warning	Keyframes past a layer's clip window never play; has `layerId`
`frozen_video`	info	Picture unchanged for ≥ `minFrozenSec` (2 s); fine for held title cards
`audio_silence`	info	≥ `minSilenceSec` (2 s) below `silenceDbfs` (-60)
`audio_silent`	info	The whole track is silent

passed is false when any error or warning is present. qa.audio.integratedLufs is ITU-R BS.1770 loudness (streaming platforms normalize to about -14 LUFS); qa.audio.peakDbfs is the sample peak.

Inspecting a Finished File with ffmpeg

For files rendered elsewhere, or to compare against a previous render, inspect with ffmpeg/ffprobe:

# 1. Check video and audio stream properties
ffprobe -v error -show_entries stream=codec_name,width,height,duration,r_frame_rate output/final-video.mp4

# 2. Detect unexpected black frames or dead transitions
ffmpeg -i output/final-video.mp4 -vf "blackdetect=d=0.5:pix_th=0.10" -f null -

# 3. Verify audio loudness levels (prevent clipping / silence)
ffmpeg -i output/final-video.mp4 -af "volumedetect" -f null -

Common Rendering Failure Modes

Symptom	Root Cause	Solution
Entire frame renders black	Missing font registration or `NaN` in uniform	Ensure every font used in `Layer.text` is registered via `FontManager.register()` and listed in `comp.fonts`. Verify all signal outputs are finite numbers.
Media layer is invisible / blank	Bad input handle binding or zero opacity	Check that the media path exists and `opacity > 0`. If using an effect or section, ensure `fit: "cover"` and valid container dimensions.
Audio cuts out early	Missing `durationFrames` on audio layer	Explicitly assign `durationFrames: totalFrames` on `Layer.audio()`.
Vision cutout / mask is empty on a single `renderFrame`	Vision reads the previous rendered frame; frame 0 has none	Render two or more consecutive frames with the same `HeadlessMediaRenderer` and keep the last. `renderVideo` handles this automatically.
Ghosted double of the subject in a vision cutout	Frames rendered out of order (frame grid, or jumping from frame 3 to 48), so the cutout uses pixels from another frame	Verify vision layers with sequential frames or the exported MP4, not `renderFrameGrid`.
Script hangs after rendering	Background worker thread pool held open	Always include `process.exit(0)` at the end of render scripts.
Text looks misaligned or truncated	Layout node box smaller than text string	Set explicit wrapping `width` (e.g. `width: 1400`) or use `width: "fill"`.
Colors appear washed out / dark edges	Unpremultiplied alpha blending	The engine operates with premultiplied alpha. Ensure custom shaders use `srcFactor: "one"`.