SuperAPI AI Directory
Partner中文AI newsFavorites
Skillse2e-test-healer

e2e-test-healer

Testing & quality

View repository

Updated 10-15 21:02

About this skill

E2E Test Healer

Take a failed E2E test and a healing loop that fixes itself until it passesoperate it. Rather than simply turning a red test green, find the root cause using the evidence left in the trace and judge "whether the test is wrong / or the product code is broken" and fix the correct side.

This skill is not a Playwright usage tutorial. Mechanical tasks such as launching the browser and navigating traces are delegated to the existing playwright-cli skill, and here we focus on what, why, and when to look at — the diagnostic judgment.

Key insight: trace = everything that happened at that time

When Playwright runs a test, it leaves a trace (zip). Inside it, everything that happened during the run is contained in its entirety.

  • The actions of each step (click/fill/goto …) and their execution order
  • Per-step screen snapshots (DOM + screenshots)
  • All network requests/responses (headers, bodies, status codes, timing)
  • Console logs and error stacks

The key point is that you do not read all of it at once. Just as a developer opens the console tab when needed and the network tab when needed in the browser devtools, the agent also only the information that confirms the hypothesis, after forming a hypothesis, pulls it out of the trace. A trace is "the entire crime scene," and diagnosis is the work of narrowing down the clues at the scene.

The specific commands to open a trace (tracing-start/stop, trace zip navigation) should not be memorized directly; instead, read playwright-cli의 references/tracing.md at that moment and use it.

Healing loop (run → debug → root-cause → fix → verify → repeat)

Run the sequence below as one cycle, and until it becomes green repeat. However, "passing" is not the same as "solved" — do not skip the judgment in step 5.

1. Run — reproduce the failure

First, check whether the failure is real and reproducible. A failure that cannot be reproduced cannot be fixed.

  • Run only the specified spec with tracing enabled. Do not run the entire suite (it is slow and the signal gets buried).
  • Obtain the failure message, exit code, and trace artifact path.
  • If it passes on the first try, that is a flaky signal. Repeat the same spec several times to see the reproduction rate (e.g., --repeat-each). For flaky diagnosis, references/diagnosis-playbook.md.

For command recipes (including project script discovery), references/loop-recipes.md.

2. Debug — narrow down the clues in the trace

Do not guess from the stack trace alone; open the trace. The tab to look at differs depending on the symptom — narrow down based on hypotheses, just like using devtools.

  • Where did it stop → just before the failure Step snapshot/screenshot. Was the expected screen displayed?
  • The action itself failed (locator timeout, etc.) → at that point DOM snapshot. Was the element absent, obscured, or not rendered yet?
  • Data/state is abnormal → Network tab. Was that API 4xx/5xx, did the response body differ from expectations, or was it not called at all?
  • JS exception/white screen → Console logs/error stack.

The symptom→trace-tab mapping table is in references/diagnosis-playbook.md. If you get stuck, start by reading that.

3. Root-cause — put the three things side by side and look at them

This is the most important step. Don't draw conclusions from the trace symptoms alone; triangulate the three sources. triangulatedo it.

  1. Test code — do the selectors, assertions, waits, and assumptions match the actual UI?
  2. Actual (product) code — what does the component/page/API handler that the test calls actually do?
  3. git diff — what did the most recent change touch? Narrow down by timing whether the breakage is due to a code change or a test change.

Here, you must determine the branch:

근본 원인이 어느 쪽인가?
├─ 제품 코드가 깨졌다(회귀)        → 제품 코드를 고친다. 테스트는 건드리지 않는다.
├─ 테스트가 낡았다/틀렸다           → 테스트(셀렉터/단언/대기/픽스처)를 고친다.
└─ 둘 다 / 모호하다                → 멈추고 증거와 함께 사람에게 확인받는다.

For the details of the determination (the signals that tell which side it is), read references/decision-test-vs-code.md.

Never do this: loosen assertions without knowing the cause, delete expector insert infinite waitForTimeoutto just make it green. That is covering up a regression with tests and hiding it. The reason tests exist is to catch breakage.

4. Fix — minimally fix only the side you determined

  • Modify only the side decided in step 3. Do not touch code unrelated to the cause (surgical).
  • If it's the test side: fix it with stable selectors (role/label based), explicit waits (no arbitrary sleeps), and deterministic fixtures.
  • If it's the product code side: fix the change that caused the regression. If possible, strengthen the test with assertions that pin down this bug.
  • Leave a one-line reason for the fix (why this is the root cause). It becomes the basis for later verification and review.

5. Verify — return to the same trace loop

After the fix, rerun the same spec with trace .

  • Still red → go back to step 2. The new trace shows new symptoms. Update the hypothesis and narrow it down again.
  • Green → verify that it is truly green:
    • If it was flaky, confirm stability by running repeatedly (passing once ≠ resolved).
    • Reconfirm that the assertion is actually checking something meaningful (that you didn't weaken it just to make it pass).
  • If it is stably green, the final gate: for the changed files, run lint와 type-check(in this repository, yarn lint / yarn type-check, RULE-BUILD-002) to check that there is no regression.

Loop exit condition

Stop if any of the following applies:

  • If the spec is stably green + lint/type pass → done. Summarize in one line what the cause was and where you fixed it.
  • If the cause is ambiguous between test and code → stop and ask a human with evidence (what you saw in the trace, the diff, candidate causes).
  • With the same hypothesis, no progress after two or three fixes → do not run more loops; organize the blocked point, what you tried, and trace evidence, and report. Do not break the tests with an infinite loop.

Antipatterns (stop if you are doing this)

  • Fixing by guessing from only the stack trace without opening the trace.
  • Weakening/deleting assertions or increasing sleep to make it green when the cause is unknown.
  • Twisting the tests to hide a regression when the product code is broken.
  • Changing tests and product code wildly at the same time in one cycle, making it impossible to know what the cause was.
  • Reporting "fixed" without confirming it passes (especially for flaky tests).
  • Not running lint/type at the end and letting new breakage slip through.

In a large-scale Next.js repository (practical notes)

  • Instead of the whole suite, a single spec/projectrun only that with trace to keep the feedback loop short.
  • Be conscious of the boundary between test code and product code: this repository is a 5-layer Clean Architecture. Product-side regressions usually occur in presentation/features/*View, application/hooks/*, infrastructure/api/*(the project's AGENTS.md or .codex/rules/**'s architecture rules).
  • What network clues point to: compare the failed request in the trace → infrastructure/api/{domain}/services.ts's calls.
  • Match the verification gate to this repository's commands: yarn lint, yarn type-check(RULE-BUILD-002). Check the test runner script in package.json.

Reference files (read when needed)

  • references/diagnosis-playbook.md — symptom → which trace tab/information to look at mapping, flaky diagnosis.
  • references/decision-test-vs-code.md — signals and examples that distinguish "the test is wrong vs the product code is broken."
  • references/loop-recipes.md — run/reproduce/verify command recipes, project script discovery, lint/type gate.
  • (external building block) playwright-cli skill — browser driving,tracing-start/stop·trace zip exploration follows that skill's references/tracing.md. Do not duplicate the explanation here.