web-clone

Web Clone · Website Reproduction Methodology

Turn "reproducing a website" into a repeatable process. In OpenDesign, by default work in the current project directory:NOTES.md、RECON/、CLONE_REPORT.md、CLONE_AUDIT.md and the final previewable index.html should both be written inside the current project, unless the user explicitly specifies an external working directory.

OpenDesign environment preparation (before running any scripts/ , check first)：

• This skill's scripts will be staged into the project at .od-skills/<插件目录>/scripts/(the exact path is in the skill preface). The node scripts/xxx.mjs written in the commands in this document is resolved according to that path, for example node .od-skills/<插件目录>/scripts/recon-site.mjs ...；RECON/、assets/ and other artifacts are still written to the project root.
• The script has a built-in zero-dependency control layer based on Chrome DevTools Protocol, directly reusing the system's existing Chrome, Edge, or Chromium; no need to start an Electron client, and it also supports pure od CLI. Inside the Agent sandbox, a temporary CDP session will be created through a local daemon, to avoid the sandbox blocking Chrome child processes.It is forbidden to execute npm install playwright in the user's project or download Chromium. If the precheck report has no compatible browser, directly state that a system browser is missing or specify the path via OD_BROWSER_EXECUTABLE_PATH , do not switch to visual estimation, and do not repeatedly install dependencies. If the source code development environment already provides OD_PLAYWRIGHT_PACKAGE, the script is still compatible with that runtime, but it is not a prerequisite for the product pipeline.

Rule number one: real source code is supreme, never trust AI-inferred code

Any AI-generated "reproduction analysis/construction blueprint",the conceptual skeleton in the main text can be referenced, but the executable code blocks inside are by default all fabricated, and must be checked line by line against the real source code, otherwise copying it will definitely break.

Evidence (marbles case): an AI analysis document turned the original site's real architecture of "analytically solving ray-sphere intersections + encoding the optical result into a displacement map, handed to SVG feDisplacementMap to distort the real DOM" into a fabricated "ray-marching + SDF + sampling the DOM as a texture"—two completely different implementations; copying it will not produce the original effect and will be N times slower. See references/marbles-case.md。

So the first action is always:get the real source code。

Decision tree (follow in order, no skipping)

Step 0 · First build the standard project skeleton

node scripts/init-clone.mjs <站名> --url <原站URL> --in-place

This script will create inside the current project NOTES.md、RECON/screenshots/, to avoid manually missing artifacts every time.

Step 1 · First search GitHub for the source code, don't rush to scrape the site

unset SSL_CERT_FILE   # macOS 怪癖，bash 前先解
# 按站名/产品名搜
SSL_CERT_FILE=/etc/ssl/cert.pem gh api "search/repositories?q=<关键词>" \
  | jq -r '.items[] | "\(.full_name) ⭐\(.stargazers_count) \(.description)"' | head -10
# vercel.app/netlify.app/github.io 的 URL slug 常常就是 仓库名 / 部署者用户名

• Single-file site (github.io / pure HTML) → directly fetch raw:curl -sL https://raw.githubusercontent.com/<user>/<repo>/main/index.html
• Source code found and the license permits → skip to Step 4 and clone directly. Lesson: searching GitHub first can save 30 minutes of detours.

Step 2 · No source code found → browser reconnaissance (probe)

Use the built-in zero-dependency CDP browser control layer to run probes and extract signals (framework / window.THREE / canvas count / smooth scrolling library / fonts / scrollHeight). Screenshots at 1440/768/390 three tiers + reconnaissance JSON saved to RECON/。

Prefer using the built-in script to run standard reconnaissance:

node scripts/recon-site.mjs \
  --url <原站URL> \
  --out RECON \
  --label original

# 真浏览器全程滚动，把页面真实用到的图片/字体/媒体（含第三方 CDN、防盗链资产）
# 全部拔到本地，并生成 assets/fonts/fonts.css（自托管 @font-face）+ URL→本地路径映射。
node scripts/asset-harvest.mjs \
  --url <原站URL> \
  --recon RECON/original-recon.json \
  --out assets \
  --manifest RECON/asset-manifest.json

node scripts/network-capture.mjs \
  --url <原站URL> \
  --out RECON/network \
  --label original

node scripts/route-crawl.mjs \
  --url <原站URL> \
  --out RECON/routes \
  --label original \
  --max-pages 25 \
  --max-depth 2

node scripts/interaction-probe.mjs \
  --url <原站URL> \
  --out RECON/interactions \
  --label original

node scripts/sourcemap-hunt.mjs \
  --recon RECON/original-recon.json \
  --out RECON/sourcemaps

--wait only indicates the additional stabilization wait after page navigation completes, not the navigation timeout. The navigation timeout can only use --navigation-timeout <毫秒> adjust; do not blindly retry the same error by increasing --wait .

For logged-in private sites, use the browser context already logged in within the current environment; for localhost / public sites without login, use the built-in CDP probe directly.

Step 2.5 · Grade complexity first, don't make blind promises

Based on reconnaissance results, first write a "pre-replication prediction": complexity L1-L6, recommended mode (faithful replication / visual replication / content overhaul), expected replicable scope, and the parts explicitly not cloned. For grading and scoring rules, see references/assessment.md。

Mode selection discipline (default when the user has not answered the confirmation items): when the user says "replicate/clone this website", the default isfaithful replication—fidelity is the primary goal in this scenario. Do not downgrade to visual replication, swap in placeholder brands, or replace photography with layout blocks on your own initiative just because "the original site is a commercial site / has copyrighted content": that will be directly perceived by the user as "the images weren't captured, the fonts are wrong, the colors aren't faithful." The correct approach is tofirst faithfully replicate it in full(real images, real fonts, and real color values all put in place), write copyright/trademark risks into the "pre-deployment replacement checklist" in NOTES.md, and leave replacement actions for Step 6 for the user to decide. Only when the user explicitly wants to "make my own site / swap in my brand" should you go with visual replication / content overhaul. Likewise,do not skip script reconnaissance just because "the target site is a risk-control-sensitive SPA"——recon-site.mjs/asset-harvest.mjs uses a real browser, so run it first and see; if it fails, then downgrade and record it truthfully.

If the mode is "visual replication" or "content overhaul" → also produce a structured design identity design-dna.json, turning "the feel of that site" into a versionable, comparable token specification, so that during replacement in Step 6 there is a basis for "keep the DNA, swap the content":

node scripts/dna-scaffold.mjs \
  --recon RECON/original-recon.json \
  --out   RECON/design-dna.json \
  --name  "<站名>"

The script will best-effort prefill the fonts/color candidates/framework effect signals discovered during reconnaissance, and the remaining fields should be completed manually via Analyze. For the three-dimensional structure (design_system / design_style / visual_effects), the complete schema, and applicable boundaries, see references/design-dna.md。

⚠️ Do not use DNA in the "faithful replication" branch: the real source code is the truth; do not let DNA of "approximate style" dilute the byte-for-byte iron rule.

Step 3 · Choose a path based on reconnaissance results

Reconnaissance result	Which path to take
Static HTML/CSS, no framework	wget --mirror Capture a mirror → remove tracking scripts → change copy
React / Vue / Next (content-oriented)	Rebuild the template (e.g. ai-website-cloner-template, Node 24+), feed in content
SPA / SaaS / data-driven pages	First run network-capture.mjs to save API fixtures → local JSON/mock server stand-in
Multi-page official site / product site	First run route-crawl.mjs to build a route map → extract a template for each page type → replace content uniformly
Complex interactive site	First run interaction-probe.mjs to record hover/click/scroll/canvas drag states → add interactions by state; do not capture only the first screen
WebGL / Canvas / Three.js heavy frontend	Deep reverse-engineer the real source code (see below) → faithful replication or find a similar open-source 3D template and swap content. Single-file native sites are often preserved byte-for-byte = the most faithful replication.When the real source code cannot be found, use runtime frame capture + a baseline gate; for discipline, see references/effect-extraction.md(can delegate to web-shader-extractor)
Static build sites (Astro/Vite SSG/Hugo), with heavy WebGL	mirror-site.mjs Full mirror deployment assets → self-hosted fonts + remove tracking → local web root serving = true source 1:1 faithful reproduction. For static sites, "getting the true source" = "mirroring the entire set of deployment assets". See the recipe at references/static-mirror.md. Example: oryzo.ai (Lusion, L6, Gaussian splatting, hero pixel diff 5/5)
Sites using ready-made open-source themes (Astro/Hugo themes)	Go to the corresponding theme marketplace to find the source theme (only for sites applying ready-made themes; for custom sites, use the full mirror from the previous line, don't come to this line)

L4-L6 complex sites follow references/complex-playbooks.md , don't just use the ordinary official website process.

Step 4 · Set up the project in the current project

# 当前 OpenDesign 项目目录就是复刻工作区。
pwd
# git 源码：clone 到 source/ 或直接放入当前目录；单文件：放进来。原始源码留一份只读基准 index-original.html
# 检查 Node 版本（package.json engines），nvm use 对应版本，钉 .nvmrc

Step 5 · Remove tracking + write metadata + verify

• Remove tracking：Google Analytics（gtag / googletagmanager), pixels, heatmaps—cut out precisely line by line (GA blocks are often at the top of <head> ).
• OpenDesign preview adaptation: before delivery, you must change the project root asset references to relative paths, to avoid /reference-assets/... hitting the OpenDesign app root in file preview and causing bare HTML:

node scripts/od-preview-rewrite.mjs --project .

• Write NOTES.md(required): include complexity, reproduction mode, original site vs clone site comparison, fidelity score, known gaps. See the template at references/deliverables.md。
• For complex sites, write TEARDOWN.md(technical breakdown). Mark all conclusions with true source line numbers.
• Score after reproduction: according to references/assessment.md , score structure / visual / interaction / responsive / content replacement / functional completeness. Scores must be supported by screenshots, source code, and runtime results.
• Real browser verification(hard requirement, not allowed to just look at the code and say "it should run"): start a local server → open in browser → capture console (no JS/WebGL compilation errors) → screenshot and compare with the original site. Honestly record the parts that cannot be verified (e.g., synthetic PointerEvent isTrusted=false cannot trigger drag, write it truthfully, don't fake "drag succeeded").

After reproduction is complete, run clone site reconnaissance once more and generate an automatic comparison report:

node scripts/recon-site.mjs \
  --url http://127.0.0.1:<端口>/ \
  --out RECON \
  --label clone

node scripts/route-crawl.mjs \
  --url http://127.0.0.1:<端口>/ \
  --out RECON/routes-clone \
  --label clone \
  --max-pages 25 \
  --max-depth 2

node scripts/interaction-probe.mjs \
  --url http://127.0.0.1:<端口>/ \
  --out RECON/interactions-clone \
  --label clone

node scripts/compare-recon.mjs \
  --original RECON/original-recon.json \
  --clone RECON/clone-recon.json \
  --visual-diff RECON/visual-diff-1440.json \
  --original-routes RECON/routes/original-route-map.json \
  --clone-routes RECON/routes-clone/clone-route-map.json \
  --original-interactions RECON/interactions/original-interactions.json \
  --clone-interactions RECON/interactions-clone/clone-interactions.json \
  --out CLONE_REPORT.md

node scripts/visual-diff.mjs \
  --original RECON/screenshots/original-1440.png \
  --clone RECON/screenshots/clone-1440.png \
  --out RECON/visual-diff-1440.json \
  --diff RECON/screenshots/visual-diff-1440.png

# --recon + --strict: 字体/图片/颜色保真硬门槛，有硬伤 exit 2 —— 修完重跑，
# 不通过不许交付。
node scripts/audit-clone.mjs \
  --project . \
  --brand "<原站品牌名>" \
  --recon RECON/original-recon.json \
  --strict \
  --out CLONE_AUDIT.md

Asset and color fidelity (hard threshold, violation = reproduction failure)

The most common way reproduction fails is not structural error, butwrong fonts, images not downloaded, colors eyeballed. The following three are iron rules,audit-clone.mjs --recon --strict will machine-verify:

1. Fonts must be self-hosted real fonts; approximating with system fonts is forbidden. The original site's font files are almost always on third-party CDNs (Typekit / Google Fonts / brand-owned CDN), with hotlink protection—so you must use asset-harvest.mjs --url(real browser network stack) to fetch them, not bare curl. The output assets/fonts/fonts.css has already rewritten @font-face to local paths, the page directly <link rel="stylesheet" href="assets/fonts/fonts.css">, then font-family copy verbatim the value of palette.*.fontFamily / fontFaces[].family in the recon JSON. Writing -apple-system / "Helvetica Neue" fallback chain to replace the original site's custom font = directly unqualified.
2. Images must be landed as local real images; gradient/SVG placeholders are forbidden as substitutes. asset-harvest.mjs --url will scroll the full page and land lazy-loaded images, srcset variants, and CSS background images all according to asset-manifest.json(originalUrl → localPath) into assets/images/. When building the page, mechanically replace references according to the manifest; if a certain image fails to download, switch --recon fallback source or from RECON/network capture, and only when it truly cannot be obtained is a placeholder allowed, with a note in NOTES.md.
3. Colors must be copied exactly from recon's computed values; visual estimation is forbidden. RECON/original-recon.json 's palette(computed backgroundColor/color/borderColor of body/header/nav/main/footer/buttons) and rootVariables are the standard answer;original-summary.md also has a summary. When writing CSS variables, copy these values directly—if footer is rgb(17,17,17) then write #111111, and do not write a "looks about right" #0a0a0a。
4. The scroll feel must be consistent; falling back to default native scrolling is forbidden. recon's frameworks(lenis / gsap detection) + motion（htmlScrollBehavior / scroll-snap rule count / sticky·fixed element count) is the original site's scrolling recipe: if the original site uses an inertial smooth-scroll library, use the same one (Lenis, etc.) or an equivalent implementation; if there is scroll-snap then restore snap; sticky navigation, parallax, and scroll-triggered animations must be aligned one by one. During acceptance, compare against the original site using interaction-probe.mjs 's scroll-sequence screenshots, and the mid-scroll states (sticky shadow, entrance-animation trigger points) must also match.

Step 6 · Replace with the user's own content

The goal is always "make the user's own site," not to clone an identical one. The replacement trio: text (index.html/data/*.json/content/*.md), media (public/assets), brand colors (CSS variables / Tailwind theme). If the structure is non-trivial, write REPLACE_GUIDE.md.

If you have done design-dna.json(visual/heavy-remix mode): this step is its fulfillment—keep the DNA, replace the content. Turn design_system into CSS custom properties, make subjective trade-offs according to design_style , and choose the implementation tier according to visual_effects.effect_intensity (lightweight CSS / medium Canvas+GSAP / heavy Three.js); for assets, prefer using asset-harvest.mjs to fetch the original site's real images, and do not AI-redraw approximations. For the generation workflow, see references/design-dna.md。

Reverse-engineering WebGL/Canvas-heavy frontends (core craft)

Break the interactive site intotechnical pillars, locate the real implementation for each pillar + mark line numbers: rendering (WebGL/shader algorithms), compositing (SVG filter / multiple canvases / post-processing), physics, interaction, audio. Only then check any secondhand analysis.

The three-part discipline when reverse-engineering effects(to cure "polishing while picking apart, ending up neither similar nor explainable"):

1. Evidence grading: mark every conclusion as SOURCE(real source code/source-map/runtime dump/frame capture),PARTIAL(name/slice, to be proven),GUESS(visual fit/magic number).Unmarked = GUESS, and before copying it must be upgraded to SOURCE.
2. no-compensation: it is strictly forbidden to mask timing/coordinate/state errors by tweaking brightness/speed/position/noise; fitted values are still marked GUESS, and state what evidence is needed to upgrade them.
3. baseline-first gate: first use real draw calls/shaders/uniforms to make a "minimal as-is reproducible RAW REPLAY" → pass frame-by-frame comparison → only then is it allowedto refactor into an engineered project. See references/effect-extraction.mdfor details (including runtime-capture fallback + when to delegate to web-shader-extractor).

Transferable advanced paradigms(worth collecting):

• Displacement-map refraction of DOM: offscreen WebGL computes a "displacement map" with RG=displacement/B=Fresnel, then uses SVG <filter><feDisplacementMap scale=N> Use it to distort real, live, interactive HTML—what it refracts is the real DOM, and WebGL never touches DOM pixels throughout. This is the soul of marbles, and also what Three.js MeshPhysicalMaterial(transmission) cannot do (it can only make a "glass ball appearance," not "refract the entire webpage").

Detailed method + item-by-item breakdown of marbles' real architecture → references/reverse-engineering.md、references/marbles-case.md。

License and attribution (must check before cloning)

SSL_CERT_FILE=/etc/ssl/cert.pem gh api repos/<u>/<r> | jq '.license'  # + 找 LICENSE 文件 + 看 README

License	What it allows
MIT / Apache / BSD / Unlicense	Can modify, can deploy, just keep attribution
NONE (no LICENSE file / not declared)	All rights reserved by default. For local learning/replication only, must credit the original author,may not publicly redeploy without permission. Don't assume it's free to use just because the code is public
Proprietary / explicitly prohibited	Read-only learning, no copying, no deployment

⚠️ Don't equate "it's public on GitHub" or "gh api can't find it right now" with MIT—verify thoroughly.

Deliverable specifications

• Each subproject root directory:NOTES.md(source info/tech stack/license/replacement map/run commands)
• For complex interactive sites, additionally:TEARDOWN.md(technical breakdown, with line numbers)
• When reporting externally or evaluating skill effectiveness, additionally:CLONE_REPORT.md(full comparison of original site vs cloned site)
• Before going live, additionally:CLONE_AUDIT.md(tracking scripts, original site brand/language remnants, TODOs, external link risks)
• RECON/screenshots/: original site vs clone comparison image
• If the user has an external replication index, update that index as needed; within the OpenDesign project, maintaining a global hub README is not mandatory.

Built-in scripts

• scripts/init-clone.mjs: initialize the cloned project skeleton and NOTES.md。
• scripts/recon-site.mjs: use the built-in CDP control layer to open the page and scroll throughout, collecting framework/resource/DOM structure/console errors, computed colors of key sections (palette), @font-face rules and the list of actually loaded font/image resources, and save screenshots at three levels.
• scripts/asset-harvest.mjs: the real browser network stack scrolls throughout to capture and download the images/fonts/media actually used by the page (including third-party CDNs and hotlink-protected assets), generating assets/fonts/fonts.css self-hosted @font-face and an originalUrl→localPath asset list.
• scripts/network-capture.mjs: capture XHR/fetch requests and save JSON/text responses, providing local fixtures for SPA/SaaS.
• scripts/mirror-site.mjs: the real browser scrolls throughout to capture every real request → mirror same-origin assets by path (including .sog/.buf/.wasm/.riv/fonts fetched at JS runtime), for a 1:1 faithful replication of static build sites (Astro/Vite SSG/Hugo). See references/static-mirror.md。
• scripts/route-crawl.mjs: crawl internal links on the same site, saving screenshots, titles, H1s, and structural signals by route, solving the problem of only replicating the homepage for multi-page sites.
• scripts/interaction-probe.mjs: automatically perform scroll, hover, safe click, and canvas drag, saving pre- and post-interaction states, screenshots, network, and console evidence.
• scripts/sourcemap-hunt.mjs: find source maps in JS chunks, and save source mappings if available.
• scripts/compare-recon.mjs: read the reconnaissance JSON, route maps, and interaction evidence of the original site and cloned site, generating CLONE_REPORT.md。
• scripts/visual-diff.mjs: use the browser canvas to do screenshot pixel diffing, outputting a visual score and diff image.
• scripts/audit-clone.mjs: scan for tracking scripts, original site brand remnants, Japanese remnants, TODOs, and external URL risks; with --recon --strict Additionally verify that font self-hosting/image localization/key block colors match word for word; if there are hard defects, exit 2.
• scripts/od-preview-rewrite.mjs: Change project-root resource references in HTML/CSS/SVG (such as /reference-assets/main.css) to relative paths, ensuring that OpenDesign file preview and exported zip can still load resources under nested routes.
• scripts/dna-scaffold.mjs: Generate from reconnaissance JSON design-dna.json design identity skeleton (best-effort prefill of fonts/color candidates/framework effect signals), for use by the "visual replication / content overhaul" modes. See references/design-dna.md。

Capability boundaries (default scope)

• Can do with high fidelity: static marketing pages, corporate websites, content-oriented React/Vue/Next frontends, animation sites whose source code can be obtained directly.
• Can visually restore but will simplify: CMS backend data, complex scroll narratives, multi-device breakpoints, WebGL/Canvas effects, third-party embeds.
• By default does not promise a complete clone: login, payment, ordering, search recommendations, permission systems, server-side business logic, proprietary APIs, copyright-restricted assets. When needed, only make a demonstrable frontend stand-in.
• During content overhaul: prioritize preserving the original site's information architecture, rhythm, motion effects, and visual grammar, and replace the copy, images, brand colors, and business propositions with the user's own content.

Flagship case

./marbles-clone/ — a glass marble site with native WebGL + SVG Filter + self-developed physics, a byte-for-byte faithful replica + complete TEARDOWN, and an example of the "WebGL heavy-frontend branch".