SuperAPI AI Directory
Partner中文AI newsFavorites
Skillscrabpot-perf-metrics

crabpot-perf-metrics

Deployment & operations

View repository

Updated 10-16 06:10

About this skill

Purpose

Turn Crabpot/OpenClaw perf artifacts into a clear read: what changed, what is real signal, what is stale/noisy, and what action is worth taking.

When to use

  • Comparing openclaw/crabpot branches such as main, crab-beta, or crab-development.
  • Reading reports/crabpot-dashboard-data.json, reports/crabpot-import-loop-profile.json, reports/crabpot-runtime-profile.json, README dashboard metrics, or GitHub Actions report artifacts.
  • The user asks about perf, runtime, CPU, RSS, import loop, OpenClaw import/activate, dashboard deltas, or whether a Crabpot performance change is meaningful.

Inputs

  • Target repo or local checkout, usually openclaw/crabpot.
  • Branches, refs, PRs, or artifact URLs to compare.
  • Optional target OpenClaw labels/SHAs if the comparison crosses openclaw@latest, beta, or openclaw/openclaw@main.

Workflow

  1. Establish freshness before interpreting numbers.
    • Resolve branch heads and report timestamps.
    • Compare dashboard JSON with raw profile JSON on the same branch.
    • If one branch is stale, say so first and avoid strong conclusions.
    • Prefer raw JSON over README snippets when they disagree.
  2. Separate the metric families.
    • import-loop-profile: cheap cold capture loop against a tiny fixture. Good for harness/import drift, weak for production runtime claims.
    • runtime-profile: macro command timings across Crabpot analysis commands. Better trend signal for real suite cost.
    • OpenClaw lifecycle: import plus activate phases. If openClawLifecycleCount is 0, the branch dashboard does not include this signal.
  3. Interpret import-loop metrics conservatively.
    • Use p50WallMs, p95WallMs, p50PluginWallDeltaMs, p95PluginWallDeltaMs, maxPluginPeakRssDeltaMb, and maxPluginCpuDeltaMsEstimate.
    • Treat 3-run p95 as "worst of three", not a stable tail latency estimate.
    • A small wall-time bump with flat plugin RSS/CPU usually means jitter or module-resolution overhead, not a memory/CPU regression.
    • Baseline-adjusted plugin deltas are more useful than raw wall/RSS/CPU values.
  4. Interpret runtime profile metrics as the stronger trend.
    • Use summary.p50WallMs, summary.p95WallMs, summary.maxPeakRssMb, summary.maxCpuMsEstimate, and summary.maxHarnessHeapDeltaMb.
    • Inspect per-command medians for the source of change: fixture-inspection, compat-report-registry, contract-capture, synthetic-probe-plan, cold-import-readiness, workspace-plan, platform-probes, and import-loop-profile.
    • Broadly uniform slowdowns across target-aware commands usually point at target OpenClaw surface parsing or registry work, not one plugin fixture.
    • If RSS is flat and CPU moves only slightly, call it a wall-clock/runtime overhead change, not a memory leak.
  5. Account for surface changes.
    • Different fixture counts, entrypoint counts, OpenClaw labels, or commit SHAs make the comparison partly apples-to-oranges.
    • New fixtures can raise issue/probe counts while leaving perf healthy.
    • OpenClaw main vs npm latest can improve compatibility while adding modest analysis overhead.
  6. Give the user a direct read.
    • Start with the conclusion.
    • Prefer bullets and a short summary over tables unless the user asks for a table.
    • Include exact deltas and classify them: real regression, modest slowdown, noise, stale data, or missing signal.
    • Recommend one next action: refresh dashboard, add a dedicated lifecycle lane, investigate a specific command, or ignore as noise.

Useful Commands

Use gh if ghx is unavailable.

Fetch current branch artifacts from GitHub:

gh api /repos/openclaw/crabpot/contents/reports/crabpot-dashboard-data.json?ref=crab-development

Prefer a small Node or jq extraction over manual README scraping. Pull these files when available:

  • reports/crabpot-dashboard-data.json
  • reports/crabpot-import-loop-profile.json
  • reports/crabpot-runtime-profile.json
  • reports/crabpot-ci-summary.json

Outputs

  • Concise perf summary in bullets.
  • Freshness caveats when needed.
  • Important deltas with interpretation.
  • Clear owner/action recommendation, especially when lifecycle import+activate is missing from dashboard artifacts.