buddy-e2e-test

Buddy E2E Test

Actually launch the app, actually send events, and actually verify behavior. Not a simulation, not asserting on source code—it is the running app's response to external stimuli.

Division of labor with shell acceptance tests

tests/acceptance/ Coverage: build, hook script mapping, socket protocol liveness, static checks of the data model, and multi-session concurrency liveness.

This skill focuses on the gaps not covered by acceptance tests：

• permission_request complete flow (including description display, badge, and restore)
• Large payloads exceeding the socket read buffer (buffer reassembly across read() calls)
• label truncation protection (an overlong description must not crash)
• Color file corruption recovery
• Visual correctness of state transition animations

Phase 1: Dynamic discovery

Do not hardcode test parameters. Read from the source in real time:

读取 HookMessage.swift → 提取所有 HookEvent case 和 catState 映射
读取 SessionColor.swift → 提取颜色数量 = 最大并发数
读取 SocketServer.swift → 提取 socket 路径、读缓冲区大小
读取 SessionManager.swift → 提取超时值、color file 路径
读取 CatSprite.swift → 提取截断阈值、状态转换动画映射
读取 BuddyScene.swift → 提取 maxCats 值
扫描 tests/acceptance/ → 识别已有覆盖，排除重复

Compute the discovery results as test parameters:

Parameter	Source	Example
EVENT_TYPES	HookEvent enum cases	[session_start, thinking, ...]
MAX_SESSIONS	SessionColor.allCases.count	8
SOCKET_PATH	SocketServer.socketPath	/tmp/claude-buddy.sock
READ_BUFFER	SocketServer read() parameter	4096
LABEL_TRUNCATE	CatSprite showLabel truncation threshold	80
COLOR_FILE	SessionManager write path	/tmp/claude-buddy-colors.json
EXISTING_COVERAGE	Events covered by tests/acceptance/	Exclude already-tested event combinations

Phase 2: Infrastructure

# 清理残留
pkill ClaudeCodeBuddy 2>/dev/null
rm -f $SOCKET_PATH $COLOR_FILE

# 构建（必须成功，否则终止）
swift build || exit 1

# 后台启动 + 日志捕获
nohup .build/debug/ClaudeCodeBuddy > /tmp/buddy-test.log 2>&1 &
APP_PID=$!

# 等待 socket（超时 10s 则失败）
timeout 10 bash -c "until [ -S $SOCKET_PATH ]; do sleep 0.5; done" || {
  echo "FATAL: socket not ready"; kill $APP_PID; exit 1
}

Send helper:echo '{"..."}' | nc -U $SOCKET_PATH(echo automatically appends \n）

Phase 3: Scenario generation and execution

Based on the parameters discovered in Phase 1, generate the following scenario categories. Each scenariomust have an explicit verification assertion。

Category A: Basic pathway for each event type

For EVENT_TYPES ineachevent, send one valid message and verify:

• Log contains Decoded message: event=X, session=Y
• No JSON decode error
• Process is alive

Category B: Full state machine path

Covereach complete session flow composed of event types with a non-nil catState:

session_start → [每个状态至少经过一次] → session_end

Verify: complete logs + color file is created then deleted + process is alive

Category C: Filling acceptance test gaps

This is the core value of this skill. Generate scenarios for the coverage gaps identified in Phase 1:

1. permission_request complete flow: includes description → visual confirmation badge → tool_start restore
2. Large payload: description length > READ_BUFFER bytes (crosses the read boundary)
3. label truncation boundary: description exactly LABEL_TRUNCATE characters vs LABEL_TRUNCATE + 1 characters
4. Color file corruption: write garbage data then send session_start
5. EOF flush path: send without \n and then close the connection
6. Other discovered gaps

Category D: Boundary values and exceptions

• MAX_SESSIONS concurrent sessions: each is assigned a different color
• The MAX_SESSIONS + 1 th: verify eviction behavior
• Each required field missing individually: session_id, event, timestamp
• Malformed JSON + unknown event value
• Repeated session_start with the same ID

Phase 4: Validation Criteria

Layered validation (from strongest to weakest)

Layer	Method	Proof strength
V1 Log assertion	grep Exact match of Decoded message	Proves the message was received and decoded
V2 State file assertion	python3 -c Parse color file JSON	Proves the session management logic is correct
V3 Error path assertion	grep "JSON decode error"	Proves errors are handled gracefully
V4 Process liveness assertion	pgrep ClaudeCodeBuddy	Proves there is no crash
V5 Visual verification	Visually inspect the Dock area animation	The only means of visual verification for the state machine

Zero-tolerance standard

所有场景必须 PASS。没有 P2 豁免。没有"记录但不阻断"。
任何 FAIL = 测试未通过，必须修复后重跑失败场景。
App 崩溃 = 严重 BUG，立即停止测试并报告。

Verify immediately after each scenario is executed, do not save it all up for a batch check at the end.

V1-V4 are objective validation, V5 is supplementary validation

Logs can only prove that "the message was received," not that "the cat entered the correct animation." For state machine behavior (the paw animation for thinking, the red badge for permission_request, the random walking for toolUse),V5 visual confirmation must be performed in addition after log validation passes. If visual inspection is not possible (headless environment), clearly mark "V5 not verified" in the report.

Phase 5: Report

## E2E 测试报告 — {日期}
### 环境
- 构建: {debug/release}
- 源码发现: {EVENT_TYPES 数量} 个事件, {MAX_SESSIONS} 并发上限, {READ_BUFFER}B 缓冲

### 结果
| 类别 | 场景数 | PASS | FAIL |
|------|--------|------|------|
| A 基础通路 | {n} | {n} | 0 |
| B 状态机路径 | {n} | {n} | 0 |
| C 缺口补全 | {n} | {n} | 0 |
| D 边界异常 | {n} | {n} | 0 |

### 失败详情（如有）
{场景名}: {期望} vs {实际} — {日志/截图证据}

### 发现的 BUG
{编号}: {描述} — {根因} — {崩溃日志/复现步骤}

Common mistakes

Mistake	Consequence
Hardcoding the event list without reading the source code	Missing tests when new event types are added
Only checking logs without checking the color file	The message was received but session management may be broken
Checking only after sending in batches	Unable to pinpoint which message caused the problem
Skipping V5 visual verification without marking it	State machine bugs are missed
Allowing some scenarios to fail	Edge-case bugs accumulate until production crashes
Not cleaning up sessions between Waves	State contamination causes subsequent scenarios to fail spuriously