SuperAPI AI Directory
Partner中文AI newsFavorites
Skillse2e-harness-setup

e2e-harness-setup

Testing & quality

View repository

Updated 10-15 21:04

About this skill

E2E Harness Bootstrap (for AI Agents)

One-line purpose

The agent can run E2E tests write and run them on its own Lay out the four harnesses that make this possible, as if onboarding a new employee.

Why it's needed (the core insight of this skill)

Agents write code well, but "context only our team knows" is 0 they start in that state. So they don't know the things a person would have asked a colleague about in week one. Examples:

  • Authentication is not about automating the login form, but getServerState() the fact that a session is created with a server function of the type.
  • When a test dies with code 90 it's not a bug but auth expiration it's a signal, and the response is to rerun the setup project.
  • baseURL is not hardcoded but determined by environment variables (E2E_BASE_URL etc.).
  • "This flow was intentionally decided not to be tested" — past decisions.

If this context isn't written down, the agent invents plausible but wrong code anew every time. The essence of the harness is embalming tacit knowledge as explicit knowledgeto eliminate reinvention every session. This skill creates those four embalmed artifacts.

Four deliverables (harness)

  1. AGENTS.md — Full codebase guide. What the service is, how it's structured, and how the core flows, authentication, and access control work.
  2. E2E operating/writing rules SSOT (e2e/CONVENTIONS.md 등) — AGENTS.md 와 separate. Execution environment, authentication scenarios, and a record of decisions on "what is tested and what is not."
  3. Shared fixture/helpers — Shared code for repetitive tasks such as resetting authentication state (storageState fixture) and setting up test data.
  4. MCP server wiring — Connect so the agent can directly access context previously discussed in-house, such as specs, designs, issues, and PRs.

AGENTS.md 와 SSOT 를 Why separate them: AGENTS.md is the codebase map that all work (including feature development) reads, and the SSOT is the operating rules and decision log for E2E only. Mixing them makes both bloated and unread by anyone. Roles must be separated so each stays a short, living document.

Order of progress

This order is the dependency order. 1→2 is because context gathering must come before code, 3 is the stage of solidifying the authentication method discovered in 1 and 2 into code, and 4 is additional context connection, so it's last.

1. 코드베이스 정찰 → 산출: AGENTS.md          (검증: 인증·핵심 플로우·접근제어가 글로 답해짐)
2. E2E 결정 인터뷰  → 산출: SSOT 문서          (검증: baseURL·인증·만료대응·테스트 제외 항목이 적힘)
3. 공용 fixture 구축 → 산출: storageState 등    (검증: 신규 테스트가 로그인 코드 없이 인증됨)
4. MCP 배선         → 산출: .mcp.json 등        (검증: 에이전트가 이슈/PR/기획 컨텍스트를 읽음)

At the end of each step, "Can a new hire pass their first E2E test by looking only at this document/code?" consult. If it cannot, the context for that step is still insufficient.

Step 0: Identify the existing state (do not reinvent)

Before laying anything down, find what already exists. This is the surgical change principle, and it is to avoid creating a harness that conflicts with existing rules.

  • AGENTS.md, CLAUDE.md, .claude/rules/** If already exists, do not overwrite, reinforceit.
  • playwright.config.*, e2e/, tests/, *.spec.ts Read the existing patterns (project separation, baseURL, storageState path) of and follow those conventions.
  • Existing .mcp.json / Check the MCP settings.

In a large monorepo (e.g., enterprise Next.js), auth/baseURL may differ per package. Within a single SSOT, branch into a per-package table.

Step 1: AGENTS.md — codebase guide

The goal is a "service map." It lets the agent know the identity and boundaries of the test target. The items to fill in and the grounding method are → references/agents-md-guide.md.

Minimum skeleton:

# <서비스명>
## 무엇인가 / 누가 쓰나
## 아키텍처 (앱·패키지·주요 경로)
## 핵심 사용자 플로우 (E2E 가 지켜야 할 것)
## 인증 & 접근제어 (역할, 세션 생성 방식, 만료 동작)
## 로컬 실행 (baseURL 결정 방식, 환경변수)

No unsupported statements. Fill in each item by reading the actual files (middleware, auth settings, config). If you do not know, ask a person, and explicitly state any assumptions.

Step 2: E2E operations/writing rules SSOT

This is the greatest value. Single source of truth for test strategy. It contains not only operational rules but also the decision log(why this way/why we decided not to). A decision log is necessary so the agent does not propose an "approach that has already been rejected" again.

The questions that must be answered and the decision log format are → references/conventions-ssot-guide.md.

In particular, the 4 things that must not be omitted (the "context only we know" from the introduction):

  • Authentication scenario: How to authenticate. (Form automation vs getServerState() -type server session — the latter is usually faster and more stable)
  • Expiration/error signals: Whether a specific exit code/error is a "bug" or "auth expiration, rerun setup."
  • baseURL/environment: Determined by environment variables. No hardcoding.
  • Test scope: What to test and what is intentionally not done + the reason.

Step 3: Shared fixture/helper (remove repetition)

Extract repeated setup into shared code. The first and most important helper is auth state separation다.

Core pattern: perform login once in the setup project and save it as a storageState(cookie + local storage) file, and every test starts by loading that file. Since each test does not go through the login UI again, it is fast and less brittle.

  • For the exact command for saving/injecting storageState, see playwright-cli's → storage-state.md . (Do not duplicate the CLI syntax here)
  • For multiple sessions by role (simultaneous isolation such as admin/user), see → session-management.md.
  • For fixture design, test data seeding, and the re-setup pattern on expiration, see → references/fixtures-helpers-guide.md.

Security: the storageState file contains real tokens. Do not commit, .gitignore Registration, deleted after automation ends.

Step 4: Wiring the MCP Server (Connecting In-House Context)

Tests verify the expected behavior defined in planning, design, and issues. Connect MCP so the agent can directly reach that source context (e.g., issue tracker, design tool, Git hosting). That way, it solves "What does the planning document say?" on its own without asking a person.

What to connect and why, and how to write it in the SSOT → references/mcp-wiring-guide.md. For actual MCP server installation, use this repository's mcp-setup skill.

Completion Criteria

Before saying bootstrap is done, all of the following must be true:

[ ] AGENTS.md 가 인증·핵심 플로우·접근제어·baseURL 결정 방식에 답한다
[ ] SSOT 가 인증 시나리오·만료 대응·환경변수·테스트 제외 항목(+이유)을 담는다
[ ] storageState fixture 가 있고, 새 테스트가 로그인 코드 없이 인증된다
[ ] "우리만 아는 맥락" 4종(인증함수·만료코드·baseURL·범위)이 글로 박제됐다
[ ] MCP 가 연결됐거나, 미연결이면 그 이유가 SSOT 에 적혀 있다
[ ] 기존 AGENTS.md/규칙을 덮어쓰지 않고 보강했다

Final check: When only these 4 types are given to a new session agent and it is told "After logging in, verify one core flow E2E," it must be able to pass without additional questions to a person. If it cannot, trace back which harness is empty.

Reference Files