aip-composition

HyperFrames for an AI Producer project

HTML is the video: a composition is an HTML element with data-* timing attributes and one paused GSAP timeline the player drives. AI Producer's editor plays your HTML with its own player, and the export renders it with AI Producer's own pinned HyperFrames; they are two programs with separate playback and editing contracts. This page is the interface those two read, nothing more. What to draw, how it moves, where captions sit and what a moment looks like are your decisions.

What AI Producer accepts

The project lives under render-engine/: index.html, compositions/*.html, and assets under public/. AI Producer stages public/source.mp4 (the recording) and public/source.mp3 (its audio) and puts GSAP at public/vendor/gsap.min.js; load GSAP from that path.
The accepted files and upload limits are described in the AI Producer workspace format. Scripts from outside the project are refused, so a composition's logic is inline.
Split each moment by what it draws. Text, labels, cards, counters, and anything the user may want to retype or restyle are DOM elements animated by GSAP, given real depth with CSS 3D (perspective, 3D rotation, translateZ stacking) rather than laid flat. A 3D object or space, a shader-driven surface, or data too dense for DOM is drawn on a <canvas> from script - Three.js for 3D - under code-drawn layers: the commit refuses a canvas that runs on its own clock or holds a WebGL context outside its window. One moment can mix both, a canvas scene with DOM labels over it. The editor treats a canvas as one leaf, so the user cannot select or restyle what it draws; keep anything they may want to edit in DOM.
A user's image is staged at public/images/<name> and a user's clip at public/videos/<stem>.mp4, <name> being the filename reduced to ASCII letters, digits, -, _ and . (a space becomes -, anything else is dropped), because other characters change the URL the browser asks for. The hand-back check confirms every file a composition references through src or href is in the tree; a file referenced any other way (data-pip-src, a CSS url(...)) is not seen by it, so stage it in the same upload batch as the composition and confirm it in the listing after the commit.

What the editor recognises in index.html

The root: <div id="stage" data-composition-id="finecut-root" data-start="0" data-duration="<total s>" data-width="W" data-height="H">. Place this root directly in the document body, not inside a <template>. The id is fixed: the editor seeks the root timeline as window.__timelines["finecut-root"], so a root transition lives on that timeline, and the root creates the registry before any composition registers:

<script>
  window.__timelines = window.__timelines || {};
  const rootTl = gsap.timeline({ paused: true });
  ...root tweens (never a speaker zoom or filter; see below)...
  window.__timelines["finecut-root"] = rootTl;
</script>

The speaker: <video id="speaker" class="clip" src="public/source.mp4" muted playsinline data-volume="0"> and <audio id="speaker-audio" class="clip" src="public/source.mp3" data-volume="1">. Keep unique media IDs. Speaker video is silent; the source audio is the audible leader. Generic extra <audio class="clip"> tags are not AI Producer sound-effect tracks: AI Producer audio tracks require its editing document and derived track-audio elements. After a cut, use N video/audio pairs over the same two files. Each element has class="clip speaker-clip", preload="none", a unique id, and data-start, data-duration, data-media-start, and data-track-index. The player loads the active pair and warms the next pair; eager preloading repeats requests for the same source. A pair shares its data-hf-id and timing; give each pair a distinct data-hf-id. Keep speaker and speaker-audio on the first pair. DOM order is output order on each speaker track, so write the spans in playback order with consecutive output starts. The EditingScript AV track must carry the same clip IDs and source/output times; the workspace helper synchronizes it and caption timing before publication.
A visual moment: one <div class="visual-host clip" data-composition-id="<id>" data-title="<short name>" data-composition-src="compositions/<file>.html" data-start data-duration data-track-index data-width data-height> per moment. The host div is the moment the editor shows and lets the user move; content written straight into the root is not a moment. A moment is its own layer over the footage: what its composition draws sees none of the footage beneath it, so an effect that combines with what is under the moment, such as mix-blend-mode, goes on the host div in index.html, where it applies to the whole moment in the editor and the export. data-title is a few words naming the effect (for example Commute title card); the editor timeline labels the moment with it and shows the composition id when it is absent. It does not change the render or the export. In a project whose captions the service built, the host may carry the caption policy for its window: data-hide-captions="false" keeps the running caption on under the moment (absent or "true" hides it, the default), data-caption-position="<integer percent of frame height>" moves the caption band's anchor for that window, and data-caption-ink="#<six hex digits>" gives the caption type that colour under the moment, for a ground this composition painted light. The editor, the service's edit paths, and the export re-derive that caption layer's windows from these attributes, so a moment moved or deleted carries its policy with it. Caption moments you write ignore them: each cue is placed and coloured in its own composition.
Captions: caption moments, ordinary visual moments whose ids are captions-<n>, one per stretch of speech, each cue static text in its composition; the dynamic caption skill owns their timing, look and mounting. The id narrator-captions, compositions/narrator_captions.html and compositions/transcript-src.json are the service's caption layer: the editor and the export find it by that id and read its words from one inline const WORDS = [...]; array, a file declaring the id without the array is refused at commit, and only the service writes them. Never write that layer or reuse its id. A project that already has it keeps it as mounted, with no offset on its host or its .caption-band from index.html, which would move the whole full-canvas band and put every word off screen. An edit without captions needs none of these files.
A speaker seat or PIP that belongs to an editable visual moment lives inside that moment: <div class="speaker-pip-frame" data-aip-editable="speaker-pip-frame"><video id="<unique>" data-pip-src="public/source.mp4" data-start="<the host's data-start>" data-duration="<the host's data-duration>" data-media-start="<source seconds at that start>" muted playsinline data-volume="0"></video></div>, with no src. Paint an opaque moment ground over the root speaker, and animate the frame inside the child timeline when it should travel from full frame into its seat. The one exception to the timing table is that this video's data-start is the host's global start, not 0, because the export re-anchors it on a cut project by setting its media offset to that value; the editor maps it through the cut itself. Keeping the picture and payload under the same host makes a timeline move or delete apply to both. A speaker cutout the service staged through frame_speaker (public/speaker_<slug>.webm, alpha video cut to its window) is wired the same way with data-media-start="0", over a public/source.mp4 view of the same window at identical geometry; the framing skill owns when and where.
Footage other than the recording (a user clip, a stock clip): default to a root video clip on its own track in index.html for both PIP and split, <video id="<unique>" class="clip" data-track-index="<n>" src="public/videos/<stem>.mp4" data-start data-duration data-media-start muted playsinline data-volume="0">, with the moment that frames it (a mask, a frame, a title) drawing over it. A nested alternative is the same tag inside the moment's composition, <video id="<unique>" class="clip" src="public/videos/<stem>.mp4" data-start="<the host's data-start>" data-duration data-media-start muted playsinline data-volume="0">, timed on the host's clock like a PIP. The root clip is what the user can move on the timeline; the nested one moves with its moment.
A speaker zoom or color filter: an empty host with no composition file, <div class="visual-host clip" data-composition-id="<id>" data-aip-effect="zoom" data-no-timeline data-start data-duration data-src-anchor data-track-index="20" data-hide-captions="false" data-effect-scale="<above 1, at most 3>" data-effect-origin="<x% y%>" data-effect-in="<s>" data-effect-out="<s>"></div> for a zoom, or data-aip-effect="filter" on data-track-index="21" with data-effect-filter="<grayscale, sepia, saturate, contrast, brightness, invert, hue-rotate, or blur functions>" in place of the scale and origin. The effect ramps in over data-effect-in seconds, holds, and ramps out over data-effect-out inside the host's window, and the speaker is at identity outside it. data-src-anchor is the source second at data-start (the covering speaker clip's data-media-start plus the offset into that clip), so a cut moves the effect with the words it covers. Two zooms or two filters never overlap; a zoom may overlap a filter or a moment. Load the engine after the root script, <script src="public/vendor/speaker-effects.js"></script>; the service writes that file in the commit that first loads it, so do not upload it. The editor lists each host on the timeline and lets the user move, trim, delete, and retune it. A scale or filter tweened on the speaker in the root timeline is invisible to the editor, so a zoom or filter is never written that way.
A camera treatment beyond a zoom or filter (an impact shake, a white flash, a whip, a push, a handheld drift): a motion host, <div class="visual-host clip" data-composition-id="<id>" data-aip-effect="motion" data-no-timeline data-start data-duration data-src-anchor data-track-index="<900 to 907>" data-hide-captions="false" data-effect-motion="<name>" data-effect-params='{"<param>":<number>}'></div>, and one inline <script data-aip-motion="<name>"> in index.html that assigns window.__aipMotions["<name>"] = { label, params: { <param>: { min, max, default, label, unit } }, frame(t, d, p, rand) }. frame returns any of x and y (pixels), scale (1 to 3), rotate (degrees) and overlay: { color: "#rrggbb", opacity } (a full-frame color over everything, captions included), computed from its arguments alone: t is seconds into the host, d its duration, p the parameter values, and rand(i) a fixed phase per host. Never read Math.random, Date, performance, crypto, Intl or a timer anywhere in the script, and never set another name's entry: the service refuses either, because the preview, the export and each render window would draw different frames. Motions that overlap in time sit on different tracks and add up; offsets stay within 160 px and rotation within 8 degrees in total, and the service scales the picture just enough to keep its edges covered, or weakens a motion that 2x cannot cover. Declare every value a user should tune as a parameter (at most 8, each named in lower camel case of at most 16 characters, with values within +/-1,000,000); the editor shows each one as a field. Worked definitions: motion recipes.
An element the user should move and restyle as one object carries data-aip-editable="<token>" where the token is one of that element's own classes; the editor also treats .list-item, .glass-card, .media-card, .speaker-pip-frame and .item-bar as such objects. Anything else inside a moment is reachable only as text or image leaves.

Do not implement a seat or PIP owned by a visual moment as absolute-time geometry tweens on the root speaker. Moving or deleting the host does not rewrite arbitrary root-timeline tweens, so that shape would remain at its old time. Root speaker geometry is only for a camera treatment that is not owned by an independently editable visual moment, and never for its scale, its filter or a per-frame camera move: a speaker zoom or color filter is an effect host, and a shake, flash, whip or push is a motion host. The optional PIP example shows the effect-owned full-frame/PIP transition.

A sub-composition file

<template>
  <div data-composition-id="<id>" data-width="W" data-height="H">
    ...content, with its <style> and <script> inside the template...
    <script>
      const root = document.querySelector('[data-composition-id="<id>"]');
      const tl = gsap.timeline({ paused: true });
      ...tweens on root.querySelector(...) targets...
      window.__timelines["<id>"] = tl;
    </script>
  </div>
</template>

The inner div's data-composition-id equals the host's. Bind the root exactly as the snippet does, with [data-composition-id="<id>"] and nothing else in that selector, and reach every element through descendant queries on that root. The export mount strips data-composition-id and the other data-* attributes from the inner div, so at export the host is the only element carrying the id: a selector that also requires a class, an id, an attribute, or :not(.visual-host) on the same element, or a :scope > child selector, matches nothing at export, the script throws before it registers, and the export refuses the moment. Scope styles and element queries to the composition's content so multiple moments can coexist in the same document. Keep the composition root visible at its base pose; animate an inner wrapper or its children, not the composition root or host. The editor mounts only what is inside <template>, and the export mounts the template itself: a <script> written after </template> is not part of the moment in either, so write nothing after </template>; the file needs no mount script of its own. Timeline time 0 is the host's data-start, and the player seeks the timeline rather than playing it. Create and register the paused timeline synchronously after its DOM exists. Render-critical changes belong on that timeline, not in timers, requestAnimationFrame loops, playback callbacks, or CSS animations. Do not call tl.play() or start media playback yourself; the player owns the clock. Keep animation repeats finite and randomness seeded so a seek reaches a deterministic state. The host attributes own the duration and active window; do not add empty tweens to set duration or manually nest the child timeline into the root timeline.

Timing attributes

attribute	on	value
`data-start`	every clip	seconds, relative to the parent composition (a PIP or footage clip inside a composition: the host's global start, see above)
`data-duration`	every clip	seconds, the clip's own length
`data-track-index`	every clip	integer; clips on one track cannot overlap
`data-media-start`	video and audio	offset into the source file, seconds
`data-volume`	registered media	playback gain, 0 to 1; does not enroll arbitrary audio in AI Producer mixing
`data-width`, `data-height`	compositions	the canvas, px

Preview acceptance

The AI Producer skill owns delivery and the no-inspection stopping condition. Keep this contract separate from creative direction. A successful local animation sample or accepted upload does not establish AI Producer editor or export correctness.