SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillskubo-deploy-e2e

kubo-deploy-e2e

Deployment & operations

View repository

Updated 10-15 20:12

About this skill

Kubo IPFS Cluster deployment e2e / demo runbook

ClusterInfrastructureFull-chain live test:Bring up the stack (3-node cluster + Caddy) → upload via the cluster proxy → automatic multi-replica across three nodes → gateway rendering → /artifact friendly path → still readable after stopping a node → tear down. All with real curl / ipfs-cluster-ctl assertions, not relying on "it should work".

This isdeployment e2e— verifying "whether the cluster itself is correct". Agent publish pipeline（:9097 token write entry + skills/publish-artifact/publish.sh + default expiry) for an independent e2e see kubo-publish-e2e skill / make publish-e2e。

This project no longer has v1 single-node; it is unified on Cluster; "single-node" = a cluster running 1 node (see docs/MULTI_HOST_DEPLOYMENT.md §3.4). This runbook targets thesingle-host three-containertrial form (docker-compose.cluster.yml）。

Conventions and prerequisites

REPO=$(git rev-parse --show-toplevel)
COMPOSE="docker compose -f docker-compose.cluster.yml"   # 单机 3 节点 + Caddy
PROXY=http://localhost:9095     # cluster IPFS 代理 —— Agent 上传打这里
GW=http://localhost:8080        # ipfs0 原生网关 —— /ipfs/<CID>
ART=http://localhost:8088       # Caddy 反代 —— /artifact/<CID>（重写到 /ipfs）
CTL="docker exec cl-cluster0 ipfs-cluster-ctl"           # 集群管理
  • Image version pinning:kubo v0.42.0 / ipfs-cluster v1.1.6 / caddy:2-alpine。
  • Secrets:.env(CLUSTER_SECRET) and runtime/private/swarm.key(private network), both gitignored. If missing, the script will stop you.
  • Runtime data is in runtime/(entirely gitignored), survives restarts; onlydown -v clears it.
  • Docs: for deployment see docs/SINGLE_HOST_DEPLOYMENT.md, for onboarding see docs/AGENT_INTEGRATION_GUIDE.md, for boundaries/ops see docs/CAPABILITIES_AND_OPERATIONS.md。

0. Bring up the stack

⚠️ Must check port occupancy before bringing up the stack：8080/9094/9095/8088 If it is occupied by another local container (e.g. a previous single-node kubo-poc, or another service),up will report port is already allocated. First docker ps to take a look, and stop any conflicts first.

cd "$REPO"
make secrets          # 幂等生成 .env + swarm.key（已存在则跳过）
$COMPOSE up -d        # 起 3×(kubo+cluster) + caddy

# 等 cluster 代理就绪（最多 ~150s；首次要拉镜像会更久）
for i in $(seq 1 150); do curl -fsS -X POST "$PROXY/api/v0/version" >/dev/null 2>&1 && { echo "proxy ready"; break; }; sleep 1; done

# 等三 peer 成形（应为 3）
for i in $(seq 1 60); do n=$($CTL --enc=json peers ls 2>/dev/null | grep -o '"peername"' | wc -l | tr -d ' '); [ "${n:-0}" -ge 3 ] && break; sleep 2; done
$CTL peers ls         # 直观确认 cluster0/1/2 都在

1. One-click e2e (recommended)

Coverage: ① three peers formed ② upload via :9095 ③ replicas=3 ④ gateway rendering ⑤ /artifact path ⑥ still readable after stopping one node.

make e2e          # = ./e2e/run-cluster.sh（跑完自动 down）
make e2e-keep     # = ./e2e/run-cluster.sh --keep（保留集群继续手测）

Non-zero exit code means failure; at the end it prints PASS=N FAIL=M, and generates a self-contained HTML report runtime/e2e/<时间戳>/report.html(can be sent to people/archived directly). On failure, do not tear down the stack; follow the troubleshooting commands in §3 to check the logs.

2. Manual step-by-step (for demo / troubleshooting)

# ② 上传一个 HTML（务必走 :9095 cluster 代理，pin 才会全集群生效）
echo '<!doctype html><meta charset=utf-8><h1>HELLO_CLUSTER</h1>' > /tmp/demo.html
CID=$(curl -fsS -F "file=@/tmp/demo.html" "$PROXY/api/v0/add?cid-version=1&pin=true" \
      | grep -o '"Hash":"[^"]*"' | sed 's/.*:"//;s/"//')
echo "CID=$CID"

# ③ 副本分布：应三节点 PINNED
$CTL status "$CID"

# ④ 原生网关渲染：200 + text/html + 正文
curl -fsSI "$GW/ipfs/$CID" | grep -iE 'HTTP/|content-type'
curl -fsS  "$GW/ipfs/$CID"

# ⑤ /artifact 友好路径（经 Caddy）：等价但前缀友好
curl -fsSI "$ART/artifact/$CID" | grep -iE 'HTTP/|content-type'
curl -fsS  "$ART/artifact/$CID"

# ⑥ 故障容忍：停一个 kubo，仍能从其余节点读
docker stop cl-ipfs1
curl -s -o /dev/null -w 'after stop: %{http_code}\n' "$GW/ipfs/$CID"   # 仍 200
docker start cl-ipfs1

Browser demo: directly open http://localhost:8088/artifact/<CID>. Directory site (relative assets): use wrap-with-directory=true to upload, visit /artifact/<dirCID>/index.html, relative assets are resolved as well. See docs/AGENT_INTEGRATION_GUIDE.md。

3. Troubleshooting

$COMPOSE ps                         # 容器状态（kubo 应 healthy）
docker logs cl-ipfs0 2>&1 | tail -30
docker logs cl-cluster0 2>&1 | tail -30
docker exec cl-ipfs0 ipfs swarm peers          # kubo 私有 mesh 互联数（应 >=2）
$CTL peers ls
docker exec cl-caddy caddy validate --config /etc/caddy/Caddyfile

4. Tear down

$COMPOSE down            # 停容器，数据保留在 runtime/
# $COMPOSE down -v       # 连卷一起清（runtime/ 是 bind mount，需另删目录）
# rm -rf runtime/cluster # 彻底重置集群身份/数据（下次全新起）

Pitfalls encountered (must know)

  • Private network must disable AutoConf: Once Kubo v0.42 detects swarm.key, it will refuse to use the default mainnet autoconf URL, and the daemon crashes and restarts immediately on startup。scripts/init-cluster.d/001-config.sh already has ipfs config --json AutoConf.Enabled false。
  • cluster proxy 502 = ipfsproxy 's node address not configured correctly：ipfsproxy The component has aindependent of ipfshttp 's node_multiaddress, configuring only CLUSTER_IPFSHTTP_NODEMULTIADDRESS is not enough, the proxy still dials the default 127.0.0.1:5001 → 502. Both must point to the local kubo (in compose CLUSTER_IPFSPROXY_NODEMULTIADDRESS is already configured).
  • Don't rely on mDNS discovery in a private network: The official example's kubo connects to the public network and relies on the public DHT for mutual discovery; we swarm.key The private network cuts off this path, and mDNS multicast is unreliable on Docker bridge networks. This project switches to adeterministic approach: a seed node ipfs0 + the remaining nodes bootstrap to its PeerID (scripts/init-cluster.d done automatically). Kubo nodes must be interconnected for replicas to actually replicate.
  • Uploads must go through :9095, not some kubo's :5001: content uploaded by directly connecting to 5001 is cluster unaware and not replicated。
  • Port occupation: this project's early single-node kubo-poc occupies 8080/5001; if it or another container is running before starting the cluster, it will port is already allocated. First use docker ps to investigate and stop it.
  • e2e script set -euo pipefail + grep：grep returning non-zero with no match will cause the script to exit as well—counting-type pipelines need || true as a fallback.
  • A bash variable immediately followed by a Chinese full-width character："$pinned（…" will merge the multibyte first byte into the variable name and trigger set -u "unbound variable"; variable names must use curly braces "${pinned}（…"。
  • /ipfs The namespace cannot be renamed: the Kubo gateway is fixed at /ipfs、/ipns, there is no configuration option to change it to /artifact. A friendly prefix is achieved via Caddy reverse proxy uri replace /artifact/ /ipfs/ (caddy/Caddyfile), and this layer will also carry LB/TLS/auth/short links in the future.
  • Replica recovery vs backup: multiple replicas only guard against hardware failure; deleting a pin will be synchronously deleted across the whole cluster,does not guard against accidental deletion—backups are still needed runtime/。

Related:kubo-publish-e2e(publishing pipeline e2e) · docs/SINGLE_HOST_DEPLOYMENT.md · docs/MULTI_HOST_DEPLOYMENT.md · docs/CAPABILITIES_AND_OPERATIONS.md · docs/AGENT_INTEGRATION_GUIDE.md