web-clone

Skill: web-clone

Two honestly-different jobs. Pick the mode first.

Mode	Output	Use when	Engine
snapshot	ONE self-contained .html (exact bytes, offline)	archive / faithful copy / "save exactly the same offline"	builtin inliner · SingleFile CLI · monolith
reconstruct	clean Next.js + Tailwind codebase (editable)	"rebuild as code I can edit" / "clone the interface into code"	the ai-website-cloner-template pipeline

Mode + engine live in harness/web-clone.config.yaml (verified:false — the live-capture engine and reconstruct fidelity are the quarantined unknowns).

WHAT

Purpose and context

• Purpose: clone a website in one of two honestly-different modes — SNAPSHOT (a self-contained offline copy, exact bytes) or RECONSTRUCT (rebuild into an editable Next.js/Tailwind codebase).
• Trigger (when to use): the user wants to "clone this site", copy the design/UI + interactions of a page, mirror a layout, save offline, or rebuild the site into editable code.
• Non-goals: no promise of perfect pixel+behavior; do not clone to republish someone else's site; not page → markdown text (that is /web-crawl).

Mental model

URL hoặc page.html → chọn mode → snapshot: engine (builtin inliner · SingleFile CLI · monolith) → clone.html / reconstruct: recon → foundation → component spec → parallel builders → assembly → visual-QA diff → Next.js codebase. Mode + engine are adapters in harness/web-clone.config.yaml (verified:false).

Input and output contract

	Field	Required?	Meaning
In	mode	yes	snapshot or reconstruct — choose first
In	local page.html or live URL	yes	local → builtin inliner offline; live → needs an engine
In	engine in config	when snapshot live	singlefile-cli or monolith
Out	clone.html	mode snapshot	one self-contained file HTML+CSS+JS+images
Out	Next.js + Tailwind codebase	mode reconstruct	component spec docs/research/components/<name>.spec.md + app/page.tsx buildable

Rules and capabilities

• RULE-01 (MUST): State the mode + be honest: snapshot builtin = LOCAL resources only; faithful live capture needs an engine; reconstruct fidelity depends on extraction quality — never claim pixel+behavior perfection.
• RULE-02 (MUST): Respect copyright / Terms — clone for reference/offline/learning, not to republish someone's site.
• RULE-03 (MUST): Self-test (snapshot core): python3 harness/scripts/web-clone.py --self-test.
• Capabilities: read pages (local file or live download via configured engine); control the browser for recon; dispatch builders in parallel; write local clone/codebase files.

Failure boundaries

• Live URL whose engine is not configured → builtin only inlines LOCAL resources → partial, state it explicitly.
• JS interactions on any snapshot engine = best-effort → do not claim full behavior.
• Reconstruct: builder npx tsc --noEmit or npm run build red → failed for that merge, fix and rerun.
• Republish purpose violates copyright/Terms → cancelled.

HOW

Main workflow

Step	Type	Inputs	Action	Outputs/exit	Failure/next
W01	judgment	user request	Choose mode (mode table above) + state limits honestly	mode	save offline → B01; code editable → B02
W02	deterministic	—	Snapshot core self-test web-clone.py --self-test	rc 0	red → stop, report
W03	effect	mode	Run branch mode A (snapshot) or B (reconstruct)	clone.html or codebase	see branch
W04	judgment	output	Report results + best-effort parts (JS interactions, fidelity)	honest report	—

Details for each step (source of truth for W03, by mode):

Mode A — snapshot (faithful 1-file copy)

1. Local page already downloaded: offline, no key — python3 harness/scripts/web-clone.py inline <page.html> --out clone.html (inlines local <link> CSS, <script src> JS, images as data-URI → one file).
2. Live URL, faithful: set engine: singlefile-cli (npm i -g single-file-cli, keeps scripts) or monolith (Rust), then web-clone.py url "<URL>" --out clone.html.
3. JS interactions = best-effort across all snapshot engines (the quarantined unknown).

Mode B — reconstruct (→ editable Next.js, the ai-website-cloner-template method)

This is NOT a byte-copy — it reverse-engineers the page into a fresh, clean codebase. Method (distilled from JCodesMore's ai-website-cloner-template, ~6k★; see Related):

1. Reconnaissance — browser-MCP/Computer-Use: full-page screenshots @desktop 1440 + mobile 390; extract global design tokens (fonts, colors, favicons); sweep scroll/click/hover to find every interactive behavior; map page topology.
2. Foundation — layout.tsx fonts, globals.css color tokens + animations, TS interfaces, inline SVGs → React icon components, Node asset-download script → public/.
3. Component spec & dispatch — per section: read EXACT getComputedStyle() values (never estimate) → write docs/research/components/<name>.spec.md (the builder contract: DOM, computed styles, state transitions, verbatim content, asset paths, breakpoints) → dispatch parallel builder subagents (one per sub-component for complex sections; ≤~150 lines spec each).
4. Assembly — import components into app/page.tsx; wire scroll-snap / Lenis / IntersectionObserver.
5. Visual-QA diff — compare clone vs original @desktop+mobile, test every interactive state, fix discrepancies by re-extracting. Builders verify npx tsc --noEmit; main npm run build per merge.

Stack: Next.js + TypeScript + Tailwind v4 + shadcn/ui. Dispatch builders via overstack /orchestration or the Agent tool.

Branches

ID	Kind	Guard	Behavior	Skip / failure	Rejoin
B01	user_optional	mode = snapshot	Mode A: page local → inline; live URL → engine singlefile-cli/monolith + url	engine not yet available → local resources only, report partial	W04
B02	user_optional	mode = reconstruct	Mode B 5 phases, builders in parallel via /orchestration or Agent tool	build red → re-extract, fix	W04
B03	capability_optional	faithful live capture needs an external engine	set engine: in harness/web-clone.config.yaml	not installed → builtin	B01

Validation and stopping

Deterministic: python3 harness/scripts/web-clone.py --self-test; builders npx tsc --noEmit, main npm run build per merge. Needs review: visual-QA diff clone vs original @desktop+mobile and every interactive state. Stop when output exists and limits are clearly stated; do not loop re-extract indefinitely — report remaining discrepancies.

Examples

• Positive: "save this page exactly as offline" with page.html already loaded → python3 harness/scripts/web-clone.py inline page.html --out clone.html → a single file openable offline, CSS/JS/images inlined as data-URI.
• Boundary/failure: given a live URL but config has no engine set → only local resources can be inlined, report partial and suggest npm i -g single-file-cli; JS interaction explicitly noted as best-effort, no claim of perfection.

Related

• harness/scripts/web-clone.py + harness/web-clone.config.yaml (engine/mode adapter).
• Reconstruct method: github.com/JCodesMore/ai-website-cloner-template (the 5-phase pipeline above).
• Snapshot engines: SingleFile (github.com/gildas-lormeau/SingleFile), monolith.
• /web-crawl (page → markdown text), /computer-use + /orchestration (recon + parallel builders).