web-clone

Web Clone · Website Reproduction Methodology

Turn "reproducing a website" into a repeatable process. Companion engineering area:~/projects/website-clones/(each reproduction is a subdirectory).

Iron rule number one: real source code is supreme, never trust AI-inferred code

Any AI-generated "reproduction analysis/construction blueprint",the conceptual skeleton of the main text can be referenced, but the executable code blocks inside are by default all fabricated, and must be verified line by line against the real source code, otherwise copying it will definitely break.

Evidence (marbles case): an AI analysis document took the original site's real architecture of "analytically solving ray-sphere intersections + encoding the optical result into a displacement map, handed to SVG feDisplacementMap to distort the real DOM" and fabricated it into "ray-marching + SDF + sampling the DOM as a texture"—two completely different implementations; copying it will not produce the original effect and will be N times slower. See references/marbles-case.md。

So the first action is always:get the real source code。

Decision tree (follow in order, no skipping)

Step 0 · First build the standard engineering skeleton

node /Users/jane/.shared-skills/web-clone/scripts/init-clone.mjs <站名> --url <原站URL>

This script will create ~/projects/website-clones/<站名>-clone/、NOTES.md、RECON/screenshots/, to avoid manually missing artifacts every time.

Step 1 · First search GitHub for the source code, don't rush to scrape the site

unset SSL_CERT_FILE   # macOS 怪癖，bash 前先解
# 按站名/产品名搜
SSL_CERT_FILE=/etc/ssl/cert.pem gh api "search/repositories?q=<关键词>" \
  | jq -r '.items[] | "\(.full_name) ⭐\(.stargazers_count) \(.description)"' | head -10
# vercel.app/netlify.app/github.io 的 URL slug 常常就是 仓库名 / 部署者用户名

• Single-file site (github.io / pure HTML) → directly scrape raw:curl -sL https://raw.githubusercontent.com/<user>/<repo>/main/index.html
• Source code found and license permits → skip to Step 4 and clone directly. Lesson: searching GitHub first can save 30 minutes of detours.

Step 2 · Source code not found → browser reconnaissance (probe)

Load the Browser skill or playwright MCP, run the probe to extract signals (framework / window.THREE / canvas count / smooth scrolling library / fonts / scrollHeight). Screenshot at three tiers 1440/768/390 + reconnaissance JSON saved to <站>/RECON/。

Prefer using the built-in script to run standard reconnaissance:

node /Users/jane/.shared-skills/web-clone/scripts/recon-site.mjs \
  --url <原站URL> \
  --out ~/projects/website-clones/<站名>-clone/RECON \
  --label original

node /Users/jane/.shared-skills/web-clone/scripts/asset-harvest.mjs \
  --recon ~/projects/website-clones/<站名>-clone/RECON/original-recon.json \
  --out ~/projects/website-clones/<站名>-clone/assets/original \
  --manifest ~/projects/website-clones/<站名>-clone/RECON/asset-manifest.json

node /Users/jane/.shared-skills/web-clone/scripts/network-capture.mjs \
  --url <原站URL> \
  --out ~/projects/website-clones/<站名>-clone/RECON/network \
  --label original

node /Users/jane/.shared-skills/web-clone/scripts/route-crawl.mjs \
  --url <原站URL> \
  --out ~/projects/website-clones/<站名>-clone/RECON/routes \
  --label original \
  --max-pages 25 \
  --max-depth 2

node /Users/jane/.shared-skills/web-clone/scripts/interaction-probe.mjs \
  --url <原站URL> \
  --out ~/projects/website-clones/<站名>-clone/RECON/interactions \
  --label original

node /Users/jane/.shared-skills/web-clone/scripts/sourcemap-hunt.mjs \
  --recon ~/projects/website-clones/<站名>-clone/RECON/original-recon.json \
  --out ~/projects/website-clones/<站名>-clone/RECON/sourcemaps

Only use bb-browserfor logged-in private-domain sites; for localhost / public sites without login use the Browser skill or playwright (don't touch bb-browser, it will grab Brave).

Step 2.5 · First grade the complexity, don't make blind promises

Based on the reconnaissance results, first write a version of "pre-reproduction prediction": complexity L1-L6, recommended mode (faithful reproduction / visual reproduction / content overhaul), estimated reproducible scope, parts explicitly not cloned. For grading and scoring rules see references/assessment.md。

If the mode is "visual reproduction" or "content overhaul" → conveniently produce a structured design identity design-dna.json, turning "the feel of that site" into a versionable, comparable token specification, so that when replacing in Step 6, "keep the DNA, swap the content" has a basis:

node /Users/jane/.shared-skills/web-clone/scripts/dna-scaffold.mjs \
  --recon ~/projects/website-clones/<站名>-clone/RECON/original-recon.json \
  --out   ~/projects/website-clones/<站名>-clone/RECON/design-dna.json \
  --name  "<站名>"

The script will best-effort prefill the reconnoitered fonts/color candidates/framework effect signals, and the remaining fields are manually completed via Analyze. For the three-dimensional structure (design_system / design_style / visual_effects), the complete schema, and applicable boundaries see references/design-dna.md。

⚠️ The "faithful reproduction" branch should not use DNA: the real source code is the truth, don't let the "approximate style" DNA dilute the byte-by-byte iron rule.

Step 3 · Choose the path based on reconnaissance results

Reconnaissance result	Which path to take
Static HTML/CSS, no framework	wget --mirror Capture mirror → remove tracking scripts → rewrite copy
React / Vue / Next (content-focused)	Rebuild template (e.g. ai-website-cloner-template, Node 24+), pour in content
SPA / SaaS / data-driven pages	First run network-capture.mjs to save API fixtures → local JSON/mock server stand-in
Multi-page official site / product site	First run route-crawl.mjs to make a route map → extract a template for each page type → uniformly replace content
Complex interactive site	First run interaction-probe.mjs to record hover/click/scroll/canvas drag states → add interactions by state, don't just screenshot the first screen
WebGL / Canvas / Three.js heavy frontend	Deeply reverse-engineer the real source (see below) → faithfully reproduce or find a similar open-source 3D template and swap the content. Single-file vanilla sites are often preserved byte-for-byte = the most faithful reproduction.When the real source can't be found, go with runtime frame capture + baseline gate, discipline see references/effect-extraction.md(can delegate to web-shader-extractor)
Static build sites (Astro/Vite SSG/Hugo), including heavy WebGL	mirror-site.mjs Full mirror of deployment assets → self-host fonts + remove tracking → local web root serving = 1:1 faithful reproduction of the real source. For static sites, "getting the real source" = "mirroring the entire set of deployment assets". Recipe see references/static-mirror.md. Example: oryzo.ai (Lusion, L6, Gaussian splatting, hero pixel diff 5/5)
Sites using an off-the-shelf open-source theme (Astro/Hugo theme)	Go to the corresponding theme marketplace to find the source theme (only for sites applying an off-the-shelf theme; custom sites go with the full mirror in the previous row, don't come to this row)

L4-L6 complex sites follow references/complex-playbooks.md , don't just use the ordinary official-site process.

Step 4 · Build the project in the clone hub

mkdir ~/projects/website-clones/<站名>-clone && cd $_
# git 源码：clone 进来；单文件：放进来。原始源码留一份只读基准 index-original.html
# 检查 Node 版本（package.json engines），nvm use 对应版本，钉 .nvmrc

Step 5 · Remove tracking + write metadata + verify

• Remove tracking：Google Analytics（gtag / googletagmanager), pixels, heatmaps—cut out precisely line by line (the GA block is often at the top of <head> ).
• Write NOTES.md(required): include complexity, reproduction mode, original site vs clone site comparison, fidelity score, known gaps. Template see references/deliverables.md。
• For complex sites write TEARDOWN.md(technical breakdown). Mark all conclusions with real source line numbers.
• Score after reproduction: according to references/assessment.md score structure / visual / interaction / responsive / content replacement / functional completeness. Scores must be supported by screenshots, source code, and run results.
• Real browser verification(hard requirement, don't just look at the code and say "it should run"): start a local server → open in browser → capture console (no JS/WebGL compile errors allowed) → screenshot and compare against the original site. Honestly record the parts that can't be verified (e.g. synthetic PointerEvent isTrusted=false can't trigger drag, write it truthfully, don't fake "drag succeeded").

After reproduction is complete, run clone-site reconnaissance once more and generate an automatic comparison report:

node /Users/jane/.shared-skills/web-clone/scripts/recon-site.mjs \
  --url http://127.0.0.1:<端口>/ \
  --out ~/projects/website-clones/<站名>-clone/RECON \
  --label clone

node /Users/jane/.shared-skills/web-clone/scripts/route-crawl.mjs \
  --url http://127.0.0.1:<端口>/ \
  --out ~/projects/website-clones/<站名>-clone/RECON/routes-clone \
  --label clone \
  --max-pages 25 \
  --max-depth 2

node /Users/jane/.shared-skills/web-clone/scripts/interaction-probe.mjs \
  --url http://127.0.0.1:<端口>/ \
  --out ~/projects/website-clones/<站名>-clone/RECON/interactions-clone \
  --label clone

node /Users/jane/.shared-skills/web-clone/scripts/compare-recon.mjs \
  --original ~/projects/website-clones/<站名>-clone/RECON/original-recon.json \
  --clone ~/projects/website-clones/<站名>-clone/RECON/clone-recon.json \
  --visual-diff ~/projects/website-clones/<站名>-clone/RECON/visual-diff-1440.json \
  --original-routes ~/projects/website-clones/<站名>-clone/RECON/routes/original-route-map.json \
  --clone-routes ~/projects/website-clones/<站名>-clone/RECON/routes-clone/clone-route-map.json \
  --original-interactions ~/projects/website-clones/<站名>-clone/RECON/interactions/original-interactions.json \
  --clone-interactions ~/projects/website-clones/<站名>-clone/RECON/interactions-clone/clone-interactions.json \
  --out ~/projects/website-clones/<站名>-clone/CLONE_REPORT.md

node /Users/jane/.shared-skills/web-clone/scripts/visual-diff.mjs \
  --original ~/projects/website-clones/<站名>-clone/RECON/screenshots/original-1440.png \
  --clone ~/projects/website-clones/<站名>-clone/RECON/screenshots/clone-1440.png \
  --out ~/projects/website-clones/<站名>-clone/RECON/visual-diff-1440.json \
  --diff ~/projects/website-clones/<站名>-clone/RECON/screenshots/visual-diff-1440.png

node /Users/jane/.shared-skills/web-clone/scripts/audit-clone.mjs \
  --project ~/projects/website-clones/<站名>-clone \
  --brand "<原站品牌名>" \
  --out ~/projects/website-clones/<站名>-clone/CLONE_AUDIT.md

Step 6 · Replace with Jane's own stuff

The goal is always "make her own site", not copy an identical one. Replacement trio: text (index.html/data/*.json/content/*.md), media (public/assets), brand colors (CSS variables / Tailwind theme). If the structure is non-trivial, write REPLACE_GUIDE.md.

Have done design-dna.json(visual/heavy-makeover mode) of: this step is its payoff—keep the DNA, swap the content. Turn design_system into CSS custom properties, make subjective trade-offs according to design_style , and choose the implementation tier according to visual_effects.effect_intensity (lightweight CSS / medium Canvas+GSAP / heavy Three.js); for assets, prefer using asset-harvest.mjs to grab real images from the original site, don't AI-redraw approximations. For the generation workflow see references/design-dna.md。

Reverse-engineering WebGL/Canvas heavy frontends (core craft)

Break an interactive site down intotechnical pillars, locate the real implementation for each pillar + mark line numbers: rendering (WebGL/shader algorithms), compositing (SVG filter / multiple canvases / post-processing), physics, interaction, audio. Only then go verify any secondhand analysis.

Three-piece discipline when reverse-engineering effects(cures "polishing while still extracting, ending up neither similar nor explainable"):

1. Evidence grading: mark every conclusion as SOURCE(real source code/source-map/runtime dump/frame capture),PARTIAL(name/slice, to be proven),GUESS(visual fit/magic number).Unmarked = GUESS, must be upgraded to SOURCE before copying.
2. no-compensation: strictly forbidden to mask timing/coordinate/state errors by tweaking brightness/speed/position/noise; fitted values are still marked GUESS, and state what evidence is needed to upgrade them.
3. baseline-first gate: first use real draw calls/shaders/uniforms to make a "minimal as-is reproducible RAW REPLAY" → frame-by-frame comparison passes → only then is it allowedto refactor and engineer it. See references/effect-extraction.md(includes runtime capture fallback + when to delegate to web-shader-extractor).

Transferable advanced paradigms(worth accumulating):

• Displacement map refracting the DOM: an offscreen WebGL computes a "displacement map" with RG=displacement/B=Fresnel, then use SVG <filter><feDisplacementMap scale=N> to distort the real, live, interactive HTML with it—what's refracted is the real DOM, and WebGL never touches DOM pixels throughout. This is the soul of marbles, and also something Three.js MeshPhysicalMaterial(transmission) cannot do (it can only make a "glass ball appearance", not "refract the entire webpage").

Detailed method + item-by-item breakdown of marbles' real architecture → references/reverse-engineering.md、references/marbles-case.md。

License and attribution (must check before cloning)

SSL_CERT_FILE=/etc/ssl/cert.pem gh api repos/<u>/<r> | jq '.license'  # + 找 LICENSE 文件 + 看 README

License	What it allows
MIT / Apache / BSD / Unlicense	Can modify, can ship, just keep attribution
NONE (no LICENSE file / not declared)	All rights reserved by default. Local learning/replication only, must credit the original author,may not be publicly redeployed without permission. Don't treat it as freely usable just because the code is public
Proprietary / explicitly prohibited	Read-only learning, no copying, no deployment

⚠️ Don't equate "it's public on GitHub" or "gh api can't find it right now" with MIT—verify thoroughly.

Deliverable spec

• Each subproject root directory:NOTES.md(source info/tech stack/license/replacement map/commands to run it)
• For complex interactive sites, additionally:TEARDOWN.md(technical breakdown, with line numbers)
• When external reporting or evaluating skill effectiveness is needed, additionally:CLONE_REPORT.md(Full comparison of original site vs clone site)
• Pre-launch additions:CLONE_AUDIT.md(tracking scripts, original site brand/language remnants, TODO, external link risks)
• RECON/screenshots/: original site vs clone comparison image
• After replication, update the hub README.md index row

Built-in scripts

• scripts/init-clone.mjs: initialize the clone project skeleton and NOTES.md。
• scripts/recon-site.mjs: use Playwright to open the page, collect framework/assets/DOM structure/console errors, and save three tiers of screenshots.
• scripts/asset-harvest.mjs: download the original site's images, scripts, and styles from the reconnaissance JSON and generate an asset inventory.
• scripts/network-capture.mjs: capture XHR/fetch requests and save JSON/text responses, creating local fixtures for SPA/SaaS.
• scripts/mirror-site.mjs: a real browser scrolls through the entire process capturing every real request → mirrors same-origin assets by path (including JavaScript runtime-fetched .sog/.buf/.wasm/.riv/fonts), for a 1:1 faithful replication of static build sites (Astro/Vite SSG/Hugo). See details in references/static-mirror.md。
• scripts/route-crawl.mjs: crawl same-site internal links, save screenshots, titles, H1s, and structural signals by route, solving the problem of only replicating the homepage for multi-page sites.
• scripts/interaction-probe.mjs: automatically execute scroll, hover, safe click, and canvas drag, saving pre- and post-interaction states, screenshots, network, and console evidence.
• scripts/sourcemap-hunt.mjs: find source maps in JS chunks, and if available, save the source mappings.
• scripts/compare-recon.mjs: read the reconnaissance JSON, route maps, and interaction evidence of the original site and clone site, and generate CLONE_REPORT.md。
• scripts/visual-diff.mjs: use browser canvas to compute screenshot pixel differences, outputting a visual score and diff image.
• scripts/audit-clone.mjs: scan for tracking scripts, original site brand remnants, Japanese remnants, TODO, and external URL risks.
• scripts/dna-scaffold.mjs: generate from the reconnaissance JSON design-dna.json design identity skeleton (best-effort prefill of font/color candidates/framework effect signals), for use in "visual replication / content overhaul" modes. See details in references/design-dna.md。

Capability boundaries (default stance)

• Can do with high fidelity: static marketing pages, corporate websites, content-oriented React/Vue/Next frontends, animation sites whose source code can be obtained directly.
• Can visually restore but will simplify: CMS backend data, complex scroll narratives, multi-device breakpoints, WebGL/Canvas effects, third-party embeds.
• By default, no full clone is promised: login, payment, ordering, search recommendations, permission systems, server-side business logic, proprietary APIs, copyright-restricted assets. When needed, only a demonstrable frontend stand-in is made.
• During content overhaul: prioritize preserving the original site's information architecture, rhythm, motion effects, and visual grammar, replacing the copy, images, brand colors, and business propositions with Jane's own content.

Flagship cases

~/projects/website-clones/marbles-clone/ — a glass marble site with native WebGL + SVG Filter + self-developed physics, a byte-for-byte faithful replication + complete TEARDOWN, serving as an example of the "WebGL heavy frontend branch".