html-in-canvas

HTML-in-Canvas skill

A live DOM (<input>, <video>, including CSS animations) is captured into canvas pixels to freely transform and interact with an experimental API; a knowledge package covering this.

Verified 2026-08-20 — Chrome Canary, chrome://flags/#canvas-draw-element Based on Enabled. It is an experimental API, so the name, signature, and behavior differ from build to build. All the rules below are obtained by debugging actual blank screens/exceptions, so omitting them means stepping on the same trap again.

Absolute rules (with symptoms if violated)

#	Rule	Symptom if violated
1	Capture only inside the source canvas's `paint` event handler. In rAF, only `src.requestPaint?.()`request with	`InvalidStateError: No cached paint record for element` — every frame is rejected, so the screen stays blank forever
2	Three kinds of signatures `(el,x,y)` → `(el,x,y,w,h)` → `(el)` sequential try + on failure, list all three errors	the last attempt's "3 arguments required" hides the real cause
3	Hiding the source canvas is done only with `position:absolute; z-index:-1` overlap. `display:none`·`visibility:hidden`·`left:-9999px` prohibited	paint skipped → transparent (blank) capture
4	wrap the entire rAF loop in try/catch + exceptions go to a red error box within the page, and the loop continues	the loop dies silently on a first-frame exception, resulting in "just a blank screen"
5	`getImageData`/`toDataURL` prohibited (the captured canvas is tainted)	`SecurityError`
6	The source element must be a child of the owning canvas (`layoutsubtree` attribute). The effect uses the buffer pattern: capture into the source → `drawImage`copy the transform to the display canvas with	sibling/external elements cannot be captured or the paint record is not guaranteed
7	All experimental API calls must be behind a detection guard. If unsupported, instead of a demo, show only requirement guidance (Chromium 146+ with flags, DOM fallback prohibited)	instant death in normal browsers
8	For the first 90 frames (~1.5 seconds), error display is withheld even on failure (initial race)	false-positive error even though it works normally

Standard skeleton (copy to start)

var drawFn = ['drawElementImage','drawElement'].find(n => typeof sctx[n] === 'function');
var lastErrs = null, captureOk = false, paintSeen = false, failStreak = 0;

function drawCompat() {
  var tries = [['(el,x,y)', () => sctx[drawFn](root, 0, 0)],
               ['(el,x,y,w,h)', () => sctx[drawFn](root, 0, 0, W, H)],
               ['(el)', () => sctx[drawFn](root)]];
  var errs = [];
  for (var [sig, run] of tries) {
    try { run(); return true; } catch (e) { errs.push(sig + ' → ' + e.name + ': ' + e.message); }
  }
  lastErrs = errs; return false;
}
function snap() {
  sctx.reset ? sctx.reset() : sctx.clearRect(0, 0, W, H);
  if (drawCompat()) { captureOk = true; failStreak = 0; return true; }
  if (!captureOk && ++failStreak === 90) showErrorBox(lastErrs.join('\n')); // 규칙 4·8
  return false;
}
src.addEventListener('paint', () => { paintSeen = true; snap(); }); // 규칙 1
function tick(t) {
  try {
    if (typeof src.requestPaint === 'function') src.requestPaint();
    if (!paintSeen) snap(); // paint 이벤트 없는 빌드 폴백
    effect(ctx, src, t);    // 마지막 성공 촬영본으로 효과 렌더
  } catch (e) { showErrorBox(e.name + ': ' + e.message); }
  requestAnimationFrame(tick); // 규칙 4: 예외가 나도 루프는 계속
}

Interaction — 3 paths

Official main method: drawElementImage()reapply the transform returned by el.style.transformto it → the browser natively handles hit testing, focus, and accessibility. Only holds for flat placement (no distortion/perspective).
setHitTestRegions() / fireOnEveryPaint: present in the spec but may not be in the build — typeof detect, then fall back to 3.
Manual forwarding (the only way in a transformed screen):
- Coordinates: pointer → getBoundingClientRect ratio correction → fx buffer coordinates → (inverse transform of the effect) → root local coordinates
- Hit test: among the accumulated offset rects of all elements under root, the deepest element (elementFromPointis above the canvas, so useless)
- pointerdownat e.preventDefault() (so the canvas doesn't steal focus), then:

Target	Forwarding	Notes
input/textarea/select	`focus()` + `setSelectionRange(끝)`	Afterward typing·Korean IMEis handled by the browser and automatically reflected in the next capture
checkbox/radio	`click()`	focus alone doesn't toggle it
select/date/color/file	`showPicker()` (focus on failure)	An open popup is on the OS layer, so it isn't captured — it appears above the actual screen
button/a/summary	`click()` + briefly toggle the active class
contenteditable	`focus()`	Not a default forwarding target — handle directly
Scroll container	fx의 `wheel` → `scrollTop += deltaY`	Scroll position is also captured
dialog	`show()` use	`showModal()`is in the top layer → outside capture

:hoveris above the canvas so it isn't hit → toggle the hover class (.cui-hover, etc.) with the hit result and in CSS :hover, .cui-hover declare the pair, style.cursoralso specify directly.
Caret (cursor) blinking may not appear in the capture (implementation-dependent). Text reflection is certain.

three.js / WebGL integration

Pipeline: source canvas (captured with the paint model) → THREE.CanvasTexture(src) → every frame texture.needsUpdate = true.
NPOT texture: minFilter = LinearFilter, generateMipmaps = false required.
Click forwarding: Raycaster → intersectObject(mesh)[0].uv → (uv.x*W, (1-uv.y)*H) → manual hit test. If a mesh is hit, interact; if it misses, distinguish as camera drag.
three.js는 Bundle the UMD build locally as a vendor — file://ES modules are blocked by CORS. Do not load from a CDN.
Direct WebGL upload is texElementImage2D(new→old signature fallback) detect then use; if unsupported, 2D capture → texImage2D fallback.
DOM→3D reverse control (e.g., toggling lights with a checkbox) can just be done with DOM event listeners — because the DOM is real.

Common mistakes (actually occurred in this session)

var frame = mesh + function frame(t) the same var/function name collision → the Mesh is passed to rAF, "parameter 1 is not of type 'Function'". Check for same-name declarations before saving the file.
When patching by string replacement, substring contamination (when there are multiple identical code blocks differing only in indentation) → replace including a unique anchor.
When porting white-background content into a dark-theme page, text color is inherited → white-on-white. On the ported content, color specify.
If a DOM label becomes wider than its container and wraps, the entire layout calculation is thrown off → white-space: nowrap + reduce the font.
If a card becomes larger than the canvas buffer, the bottom is cut off → calculate the height based on the maximum state (e.g., all accordions expanded).

CanvasUI core (this repo's library)

Encapsulates all the above rules: core/canvas-ui.js + token CSS. New demos should use this instead of implementing it themselves:

var api = CanvasUI.create({
  mount: el, html: '<div>…진짜 DOM…</div>', width: 640, height: 400,
  effect(ctx, src, t, api) {},   // 생략 시 src 중앙 표시. 버퍼 패턴으로 자유 변형
  onFrame(t, api) {},            // 물리/파티클 갱신 (effect 앞)
  onHit(el, ev, api) {},         // true 반환 시 기본 포워딩 생략
});
// api: fx·ctx·src·sctx·root·center·toLocal(x,y)·hit(x,y)·requestRender()·destroy()·state{}

Documentation site (index.html)는 CUIDocs.register({ id, section: 'basics|official|components', name, oneLiner(한 문장·45자), code, mount(el)→api, knobs(el,api)?, renderDoc? }) register pages via the contract. When routing leaves, the shell api.destroy()Since it calls , the window listener wraps destroy to remove it.

Style principles

One explanatory sentence per page. No external CDNs (everything local). No M/B/K abbreviations for amounts or numbers (use commas or ten-thousands/hundred-millions).
Verification demos (whether video capture is possible, iframe/:visited privacy, etc.) are Display the result on screen regardless of which side it isand add the measured results to this document.