This skill covers the complete workflow for automated testing of Web applications: converting manual test cases (markmap + Schema) into Playwright specs → running and verifying → discovering bugs → outputting test reports.
Core principles:
- Get familiar with the business before writing tests — When unfamiliar with a feature, first use automation scripts to actively explore the system, understand the actual behavior, and only then start writing test code
- Ask when in doubt, do not make assumptions — When encountering anything unclear during writing, you must ask the user for clarification, and never write code based on guesses
- Each automated test case corresponds to one manual test case, and each test tests only one point
- Each test is independent(self-created data + self-cleanup)
- Page Object must be used — Writing raw locators is prohibited in formal tests; all page interactions must be encapsulated in Page Objects
Engineering conventions (scaffolding, configuration, scenario code templates, Page Object specifications) are unified in references/playwright-conventions.md— load when writing code; reference implementations of common Helpers (login, multi-session, evidence collection) are in references/helpers_reference.md; for the three types of execution snippets in the type domain (a11y scanning / visual baseline / multi-browser matrix), see Sections 12–14 of the engineering conventions; when type_scope is judged to fall into the corresponding axis, load and use them by tier
When to Use
- Given test cases (markmap / Schema), need to generate Playwright spec files and execute them
- Before writing automation, need small-scale business familiarization exploration (scouting page structure, extracting selectors)
- Bugs are discovered during automated execution, need to collect evidence and record test report entries
- Need to write new Page Objects or Helper functions
When NOT to Use
- End-to-end testing of the entire requirement (pipeline of understanding → strategy → test cases → execution → report) → use the
qaskill for orchestration - Writing manual test cases → use
test-case-writingskill - Pure API interface testing (no Web UI flow) → use
api-testingskill - Withunderstanding the system / discovering risksas the purpose of an independent exploratory testing session (charter-driven, producing exploration notes) → use the
exploratory-testingskill; the workflow of this skill only performs small-scale exploration for "scouting for writing automation" - Root cause localization, impact analysis, and regression suggestions for confirmed bugs → use the
bug-analysisskill; this skill is only responsible for collecting bug evidence (screenshots/API/console) and recording report entries - Determining regression scope after code changes → use
regression-testingskill - Unit testing → use Jest/Vitest; performance stress testing → professional tools (k6, locust); security testing → dedicated security audit (see the handoff conventions in
test-strategy)
When to ask questions (must be followed)
Core rule: ask when uncertain; better to ask more than to guess blindly. The format and adjudication rules uniformly follow ../core/clarify-pattern.md(for scenarios use "execution confirmation").
| Scenario | What to ask | Do not assume on your own |
|---|---|---|
| Element location failure | "Cannot find {element} on {page}; is the actual page structure consistent with expectations?" | Do not casually switch selectors and guess |
| Unclear operation path | "The test case says {operation X}, but there is no direct entry point on the page" | Do not piece together operation steps on your own |
| Expected behavior is ambiguous | "Expected {result A}, actual {result B}, which one should be taken as authoritative?" | Do not selectively trust one of them |
| Business rules are unclear | "What exactly is the boundary of rule {X}?" | Do not substitute common default values |
| Anomaly discovered during exploration | "{anomalous behavior} was discovered, is this expected behavior or a bug?" | Do not decide on your own whether it is a bug or a feature |
| Test case repeatedly times out/unstable | "Does {page} have long connections or polling push (WebSocket/SSE/heartbeat reporting) that causes the page to never become idle?" | Do not uniformly apply networkidle waiting; handle it according to the wait degradation ladder |
Workflow Zero: Business Familiarization (mandatory prerequisite, small-scale exploration to scout for writing automation)
When it is needed: The feature module has never been tested / an unfamiliar page route appears / a new Page Object needs to be written / test cases are available but you do not know what the system looks like
This workflow issmall-scale scouting exploration(understand page structure, extract selectors, create Page Objects), and its output serves Workflow One. For afull exploration sessionaimed at understanding the system / discovering risks (charter-driven, producing exploration notes), use
exploratory-testingskill。
Steps
- Explore page structure: log in → navigate to the target page → take screenshots → enumerate all interactive elements
- Experience the core flow: walk through the complete flow according to the test case steps, take a screenshot at each step, and record API calls
- Record findings: page navigation paths, key element selectors, API interfaces, hidden behaviors,write or update the Page Object
For the exploration code template, see references/playwright-conventions.md Section 4 (explore-*.spec.ts).
Completion criteria
- Every page involved has a screenshot
- The navigation paths between pages are understood
- The selectors of key elements are confirmed
- The actual API requests are recorded
- Hidden behaviors are discovered (hidden fields, default values, preconditions)
- The corresponding Page Object has been written or updated
Workflow One: Manual test cases → automation code
Prerequisite: Workflow Zero (business familiarization) has been completed, and there is sufficient understanding of the target feature.
Test case input and automation scope determination
The input is the test-case-writing markmap test case file produced (if it contains 测试用例.schema.yaml , consume it as well):
- Schema's
execution_model: uiandautomation.supported != notest cases → the conversion targets of this workflow execution_model: dev-collab(test cases without UI collaboration) → hand over toapi-testingor keep them for manual collaborative execution; do not force UI automationautomation_planWhen it exists (test-strategyoutput), adjudicate execution according to the user's execution strategy; when the strategy is undetermined, confirm with the user which test cases to convert to automation
Steps
- Parse test cases: extract scenarios and expectations from the markmap Markdown; for test names, continue using the test case's TC number (e.g.
TC-01-03: {用例名称}); when a manual test case lacks a number, add the number first before conversion - Select/create a Page Object: check
tests/pages/whether a corresponding Page Object already exists, and prioritize reuse - Write spec: use Page Object to encapsulate all page interactions. Before finishing each test, run thethree assertion questions: ① Is there data in the preconditions that "should not match / should be excluded"? Does the assertion cover "not present"? ② Is the trigger method of the step under test consistent with the test case specification (do not change the trigger path to work around instability)? ③ On failure, can you distinguish between "the feature is broken" and "the environment did not load"? — For judgment and counterexamples, see Section 15 of the engineering conventions
- Run verification：
npx playwright test tests/{文件名}.spec.ts；Baseline zero pass prohibits delivery— If 0 tests pass in the first run, it is considered a process cognition error (selectors/popup structure/wait assumptions do not match the real page): return to the workflow, re-check with zero targeted retries, revise and rerun until ≥1 test passes before entering subsequent steps. Delivering a spec with a zero-pass baseline is prohibited
Scaffolding, scenario code templates (CRUD / form submission / state transitions / multi-user concurrency), Page Object conventions, and spec namingAt this point load references/playwright-conventions.md(Sections 1, 5–7).
Workflow 2: Bug Exploration and Recording
On the premise of having understood the business logic, systematically verify page functionality, and discover bugs and inconsistencies.
Batch failure precondition triage: when one round of execution endswith ≥3 failures, first load
../core/triage.mdto perform four-category classification (A real defect / B asset issue / C environment / D unstable). Only category A enters the evidence collection and report entries of this workflow; <3 items keeps the single-item process unchanged.Headless pipeline scenarios (PR smoke / nightly full run / release gate): checkpoints are downgraded to pending items, artifact persistence conventions, and exit code semantics follow
../core/pipeline-integration.md(load at this point); batch failures in its feedback loop also enter the triage process above.
Responsibility boundary: this workflow is responsible fordiscovery + evidence collection + report entries. After a bug isconfirmed, root cause localization, impact analysis, and regression suggestions are handed over to
bug-analysisskill (the five extended fields such as root cause analysis in the corresponding report entry are left as TODO).
Bug Discovery Strategies
| Strategy | Inspection Method | Typical Bug |
|---|---|---|
| UI expectation mismatch | Compare test case with actual screenshot | Incorrect default value, wrong copy |
| Missing form validation | Submit invalid data | Required fields not validated, ranges not restricted |
| Network anomaly | Monitor API responses | 500 error, timeout |
| Data inconsistency | Verify by reloading after operation | Submission succeeds but data is lost |
| State transition error | Verify state through multi-step operations | State not updated, stuck |
| Console error | setupConsoleLogging(page) continuous monitoring |
Component crash, silent failure |
Bug Evidence Collection (must be executed when a bug is found)
Bug 发现 → 截图操作前 → 操作触发异常 → 截图操作后 → 收集 API/控制台日志 → 写入报告
Evidence collection code (setupBugTracking / tracker.collect) see references/playwright-conventions.md Section 8; when the general Helper is implementedload references/helpers_reference.md。
Screenshot naming uniformly uses the lowercase bug-{序号}-* prefix: manual screenshots record the two states of the process (bug-001-01-操作前.png、bug-001-02-异常.png）；tracker.collect() Automatically add a full-page summary screenshot (bug-001-汇总.png) and attach it to the HTML report. In the Bug report title, BUG-{序号} is only a display format.
Bug Report Format
Recorded in {项目名}/测试报告_{来源}_{日期}.md，entry fields and kept consistent with ../core/report-template.md §3 (the single source-of-truth template for test reports, loaded at this point), ensuring entries can be directly assembled into the final report concluded by qa ; the discovery method of this skill includes "business familiarity exploration". The execution sub-report must also include §2 execution statistics (P0/P1/P2 × test cases/passed/failed/blocked/not executed table, also per report-template); in the entries, the five extended fields such as root cause analysis are left as TODO, to be filled in by bug-analysis . Per the report-template "machine-readable summary" convention, append a machine-readable summary fragment (after placeholder substitution) to the final section of the sub-report, for qa to conclude and for subsequent regression orchestration machine-readable parsing.
Explore File Lifecycle Management
Business familiarity exploration files (explore-xxx.spec.ts) aretemporary artifacts, and follow this lifecycle:
创建 explore-xxx.spec.ts → 提取知识到 Page Object → 删除 explore 文件 + 截图
Rules
- Delete immediately after knowledge extraction: After writing the discovered selectors, API paths, and page structure into Page Objects or helpers, delete the explore file
- Screenshot cleanup: The
debug-*.pngscreenshots generated during exploration are temporary debugging artifacts; delete them after confirming they are not needed - No accumulation：
tests/The directory should not contain completed explore files. If any exist, it means the knowledge extraction step was skipped - Bug report screenshots retained：
bug-{序号}-*.pngScreenshots are part of the Bug report and are not deleted
Key Techniques Quick Reference
The method names in the table below (
createDualSession/tracker.collect/setupBugTracking/isServerCrash, etc.) are examples of this skill's built-in scaffold helpers(defined inreferences/), not general Playwright APIs—when switching projects, adapt to the actual scaffold perreferences/helpers_reference.md; do not assume these functions exist.
| Scenario | Method | Notes |
|---|---|---|
| Controlled component input fields (Ant Design, etc.) | fill() is preferred; if it does not work, then keyboard.type() |
fill dispatches an input event and works for most controlled components;keyboard.type key-by-key is the most reliable but slow |
| Confirmation dialog buttons | Locate within the scope of role=dialog (e.g., page.getByRole('dialog').getByRole('button', { name: '确定' })） |
to prevent ambiguous hits on buttons with the same name on the page; same-named "OK/Delete" buttons outside the dialog are a common source of misclicks |
| Hidden required fields | page.evaluate() set the value directly |
A common cause of blocking form submission |
| Network request verification | page.on('request') listen with |
Must be set up before the action |
| Data persistence verification | page.reload() + assertions |
Confirm the data is truly saved |
| Multi-user sessions | createDualSession(browser, roleA, roleB) |
Two roles concurrently |
| Login state reuse | createSession(browser, role) |
Cache storageState to avoid repeated UI login (Engineering Conventions Section 10) |
| Wait strategy | ① auto-wait + expect assertions → ② waitForResponse → ③ networkidle(traditional SSR/MPA only) |
Top-down priority (Engineering Conventions Section 9); fixed waitForTimeout Temporary debugging only; long-polling/heartbeat pages never go idle for networkidle |
| Parallel execution | Serial by default; enable after independence criteria are met workers |
Prerequisite: self-created data + self-cleanup + unique naming verified item by item (Engineering Conventions Section 11) |
| Flaky classification | First failure passes when rerun as-is → classified as flaky | Reruns are only for classification; before identifying the root cause, mark test.fixme(Engineering Conventions Section 11) |
| Screenshot debugging | page.screenshot({ fullPage: true }) |
Take a screenshot at every key step |
| Server crash detection | isServerCrash(bodyText) |
Detect 500/502 errors |
| Unique naming | ${前缀}-${Date.now()} |
Avoid name conflicts between tests |
| Bug evidence collection | tracker.collect(testInfo, id, desc) |
One-click collection of screenshots + API + console |
| Bug exploration prerequisite | setupBugTracking(page) |
Register API and console listeners |
Common Mistakes
| Error | Consequence | Correct approach |
|---|---|---|
Controlled component value does not take effect yet repeatedly fill() |
Test hangs or assertion fails | fill When fails, switch to keyboard.type()(see quick reference table) |
| Network listener placed after click | Request cannot be captured | Register the listener first, then perform the action |
| Do not clean up test data | Subsequent tests are affected | Delete created resources in afterEach |
| Wait with a fixed timeout | Timing is unstable | Switch according to the wait degradation ladder: ① auto-wait+expect assertion → ② waitForResponse → ③ networkidle(Engineering Conventions Section 9) |
| Do not verify persistence | Data may exist only in memory | Re-assert after reload |
| Formal tests do not use Page Object | Code duplication, difficult to maintain | Encapsulate all page interactions in Page Objects |
| Explore files are not deleted | Context pollution; AI reads a large amount of dead code every time | Delete immediately after knowledge extraction |
| Make assumptions when encountering questions | Produce unreliable tests | Ask the user when uncertain |
| Bug only screenshotted without recording API/console | Developers cannot identify the root cause | Use setupBugTracking() + tracker.collect() |
| Hard-code environment addresses/accounts | Cannot run across environments, leaks sensitive information | Put uniformly in constants.ts / .env, code only references constants |
| Treat rerunning a failure until it turns green as fine | Flaky tests mix into the main branch, CI randomly turns red | Handle according to the flaky classification process: classify first, then fix the root cause; do not increase retries to forcibly suppress it (Engineering Conventions Section 11) |
| Force networkidle on long-polling/push pages | Timeout false failure | Switch according to the wait degradation ladder to waitForResponse/assertion auto-polling (Engineering Conventions Section 9) |
| Every context goes through the login page | Execution time doubles, cached authentication state is effectively useless | Use createSession() Reuse storageState, automatically fall back to UI login on failure (Engineering Convention Section 10) |
| All test cases are permanently serial and dare not run in parallel | Full execution duration expands linearly | Gradually enable after independence verification passes workers(Engineering Convention Section 11) |
| Insufficient information in manual test cases still leads to forcibly fabricating locators and operation paths | Hallucinated automation: assertions all green but the real behavior was not tested | First pass ../core/executability.md conversion gate + workflow zero-step verification of page structure |
Coordination with other skills
- Upstream：
test-case-writingproduces manual test cases (markmap + Schema) → this skill turnsexecution_model: uiand automatable test cases into executable code; the end-to-end pipeline isqaorchestrated - Bypass: for undocumented / unfamiliar systems,complete exploration sessionuse
exploratory-testing, and its exploration notes can serve as input for this skill's verification - Downstream: root cause / impact / regression analysis after Bug confirmation →
bug-analysis; execution reports and Bug entries are aligned according to../core/report-template.md, forqafinal summary - Peer: automation of pure API test cases →
api-testing