Browser automation via Playwright MCP, existing-browser CDP, and direct CDP helpers.
When to Use
- Browser automation, UI verification, form operations, screenshots
- When you want to establish the procedure with MCP and then run it in bulk with Python
- When you want to use the login state of an existing browser via CDP
- Playwright MCP /
connect_over_cdp()is unstable and you want to switch to raw CDP WebSocket - When handling UI where normal clicks tend to break, such as modals or file choosers
Not the Best Fit
- Consider API / CLI first: If read/write can be completed with an official API (YouTube Data API, Microsoft Graph, GitHub API, etc.) or CLI, prioritize that, and switch without persisting when there are signs the UI is fragile (native file chooser / shadow DOM / multi-step wizard / modal that stops the renderer) (e.g., YouTube Studio UI →
captions.insert） - URL liveness checks are done with an HTTP client, but only the dead determination escalates to the browser. The only evidence of certain disappearance is HTTP 404 / 410. 403 is a bot block, timeout is transient, and SSL
unable to get local issuer certificateoccurs on servers that do not return an intermediate CA (browsers and .NET/schannel auto-complete via AIA, but Pythonssldoes not).Invoke-WebRequest -Uri <url> -Method GetOr verify with a browser before drawing a conclusion - For UIs that have a dedicated skill, such as PowerPoint / Loop / Dynamics 365 expense entry screens, prioritize the operation rules of the corresponding skill
- Do not receive credentials, secrets, or MFA responses in chat. Have the user handle necessary input in the browser. After user interaction, check the state only by URL (
/json/list) without reading the body of screens that may display recovery codes, tokens, or passwords. Even after a "complete" declaration, a secret display screen may remain before confirmation - Exception: If the target site supports passkey / FIDO2 / WebAuthn, it can be fully unattended with a CDP virtual authenticator. Priority order is passkey (virtual authenticator) > email OTP > SMS > physical key / biometrics. For details, refer to references/instructions/webauthn-virtual-authenticator.md Refer to
Choose Mode First
- Default to a visible, headed browser. Use headless only when explicitly requested, never as an automatic recovery path. Reuse a verified authenticated session and an owned work tab when possible; a new profile does not imply reusable authentication. When a dedicated launch is already authorized, an absent session calls for an owned launch, not an unavailable verdict.
- Preserve the OS foreground window and the user's selected tab. Do not routinely call
bring_to_front,Page.bringToFront,Target.activateTarget, or native focus/keystroke helpers. This applies to browser launch, tab creation, navigation, capture, and recovery, not just clicks. - If a necessary operation cannot avoid activation, explain why and which bounded step needs it before proceeding. Do not force focus back afterward: the user may have switched applications meanwhile. Credentials and MFA remain user-entered. When handing a step to the user, give the window title and confirm the window is on the primary monitor (multi-monitor setups hide it otherwise).
- Identify the work tab, goal, and authorized side effects. Browser access or drafting permission does not authorize sending messages, purchases, trades, cancellations, or contract changes; bind approval to the destination and exact content/action. Read-only helpers must not submit or select settings options; menu expansion is allowed, but choosing an item may auto-save. Report attempted writes, including blocked ones. Pause on user editing, target drift, or lost ownership; viewing a tab does not permit discarding it.
- Prefer API/CLI helpers for supported data operations, but preserve UI execution when the goal is a demo, UI verification, or observation of the browser workflow.
| Mode | Use when | Boundary |
|---|---|---|
| Existing headed browser + MCP/CDP | Authentication or interactive exploration matters | Verify profile, owned target, and non-activation behavior |
| Playwright CLI or saved Playwright script | A stable workflow will repeat | Reuse the session; explicitly request headed mode for new launches |
| Direct CDP WebSocket | A verified lower-level route is needed | Pin target ID; do not bypass ownership or recovery limits |
For launching an existing browser CDP, checking the profile, port drift, authentication URL encode, and taking screenshots, refer to references/instructions/cdp-existing-browser.md 。
For direct WebSocket CDP launch flags,websocket-client connections, SPA hash navigation, and virtual scroll operations, refer to references/instructions/cdp-direct-websocket.instructions.md .
Core Loop
1. Reuse a suitable script or establish the flow on an owned, verified work tab.
2. Resolve the target and expected postcondition with a scoped snapshot or query.
3. Perform the action once; wait for the expected state with a deadline.
4. Read back the durable result before retrying or moving to the next item.
Keep one working control route instead of repeatedly switching MCP/CLI/CDP. Batch independent reads and return compact results; use full snapshots or screenshots at meaningful visual checkpoints, not after every read.
For downloads, save a durable extension-bearing copy before context/runner cleanup and verify its bytes/content, not its browser-history name or icon; see download handling.
Wait for required controls or an explicit empty state. A title, route or tab-click success proves neither authentication nor that the selected view matches rendered rows. Distinguish parse failure from empty data: preserve raw text, normalize observed invisible label characters only in an extraction copy, and test explicit zero, missing fields and mismatched IDs through the collector. Never inject an expected ID to pass validation. A readiness deadline ends in unverified state, not inferred logout.
In UI verification, decide the expected state and verification method before operating. Do not judge success solely by a success toast or button press; verify with the DOM, URL, persisted list rows, API read results, or evidence such as screenshots / traces.
For settings readback, locate the semantic section/group and inspect actual roles before choosing a locator: labels may move from radios to dropdowns, and similar controls may govern unrelated scopes. Read selection from checked, aria-checked, or the displayed value. A current selection or documented default does not establish the creation-time value; require historical and unchanged-state evidence, otherwise report it unverified.
For replies/messages, exclude the composer and list previews from success checks. Capture POST status and reply/message ID when available; otherwise verify the approved body and sender appear once in the intended thread after a fresh read/reload. Preserve drafts before reload, never discard another person's input, and report unavailable network evidence as unobserved. Stale counts or a retained draft after submission are not grounds to resend.
There are cases where the API and DOM states are temporarily out of sync. Even if the API returns stale / capture failure, if the on-screen cell / row aria-label or status text indicates progress, treat the DOM as a candidate source of truth as well. Do not conclude "incomplete" from the API alone.
Decision Patterns
| Situation | Action |
|---|---|
| Flow or selectors are still unknown | Use MCP interactively |
| A stable multi-step flow recurs a second time | Reuse or parameterize a saved script when repetition outweighs maintenance; for known bulk work, validate one item and script the rest on the first run |
| Snapshot returns a ref | Use normal click/type |
| Element is visible but has no ref | Confirm visually, then use evaluate or the relevant UI fallback |
| Element is not visible | Wait for a bounded readiness condition; preserve dirty forms and do not force interaction |
| Read-only tabular extraction | Keep navigation evidence, then return compact JSON from DOM evaluation |
CLI commands alone do not remove per-action reasoning. Save a verified sequence with stable locators, input/target parameters, readback, duplicate-write protection, and resumable per-item results. Store task-specific scripts with their project and generic helpers with the owning skill; record purpose and invocation for discovery. Do not persist secrets or transient snapshot refs. See repeatable workflows.
Virtualized feeds / per-item mutations and handler-count diagnosis: see UI Fallbacks.
Do not break unsaved editor / draft tabs
Keep dirty editor tabs in place; use a separate clean work tab instead of navigating or reloading them. Never auto-accept Leave or discard changes.
CDP reporting No dialog is showing does not rule out a browser-chrome prompt. Native automation may invoke Cancel/Stay only when the owned browser process, selected-tab address, exact leave-site warning and enabled button uniquely match; otherwise stop. Never handle credential, permission or arbitrary dialogs this way.
After dismissal, verify the same target responds; preserved form values are not proof of saved settings. See unsaved-form safety and native recovery.
iframe、force click、file chooser、hidden input、evaluate+fetchは UI Fallbacks Refer to.force: true is the last resort after confirming the element's existence and visibility.
Safety Rules
Before repeatedly/bulk sending technical or security content to an external platform, the content-filter preflight is UI Fallbacks Refer to.
Failure Budget
- Before a write, choose its durable success signal and one bounded recovery. Allow the initial attempt plus at most one recovery attempt per logical operation across UI, CLI, CDP, script, and replacement-tab routes; switching routes does not reset the budget.
- A timeout is not proof of failure. Re-read authoritative state; retry only when non-application is established or an idempotency mechanism prevents duplication. If the outcome remains ambiguous, stop without replaying the write.
- Stop after two failed attempts and report target, attempts, last state, and restart condition. Waiting for normal asynchronous progress is not another attempt: use a deadline and a narrow state check, not fixed sleeps or repeated full-page snapshots.
- Do not continuously fall back from UI click to DOM mutation, keyboard, or private frontend internals. Do not operate unpublished ViewModel or upload implementations except for a single incident with explicit approval
Session endedIf a login redirect or a different context is detected, do not operate on stale DOM. After re-authentication, reacquire persistent state and resume only incomplete operations
CDP exclusive control
- Use one controller per work target. If another session owns the browser and coordination is unverified, use both a free port and a separate unused user-data-dir. A different port or profile-directory alone does not isolate a Chromium process; do not attach, restart it or copy its credentials to bypass ownership.
- Pin the verified profile/context and owned target ID; re-check URL, route, query, and item ID before each write batch. Never select the first domain match again after binding the work target.
- Before a necessary handoff, stop the owned runner and verify the current tool's detach/close semantics. Do not use
browser_closeas a generic disconnect: it may destroy tabs or the browser. Preserve dirty tabs; if safe detachment is unavailable, retain the working route or stop. - Close only explicitly owned, no-longer-needed targets after result verification. Never close the last tab of a shared browser or stop another session's runner as cleanup. When Playwright MCP is attached over CDP,
browser_closemay only detach and leave the owned tab open; re-list targets and close the owned one by ID (/json/close/<id>).
When using raw WebSocket, filter responses by CDP command id, andRuntime.enable / Page.enable enable necessary domains such as these first. For details, refer to references/instructions/cdp-direct-websocket.instructions.md Refer to.
Playwright MCP の file: rejection (http.server and the list of scenarios prone to failure is delivered via CDP Recovery and Context Refer to.
Reference Map
| Need | Reference |
|---|---|
| Existing browser CDP, profile, port drift, screenshot capture | references/instructions/cdp-existing-browser.md |
| Raw CDP WebSocket | references/instructions/cdp-direct-websocket.instructions.md |
| CWS submission from an authenticated browser without CDP | references/instructions/cws-store-submission.instructions.md |
| WebAuthn virtual authenticator / passkey | references/instructions/webauthn-virtual-authenticator.md |
| CDP recovery, context selection, Windows UI Automation without CDP | references/instructions/cdp-recovery-and-context.md |
| Azure Portal iframe / OOPIF | references/instructions/azure-portal.md |
| Angular Material forms | references/instructions/angular-material.md |
| Hidden upload, VS Code Web, evaluate+fetch, UI fallback | references/instructions/ui-fallbacks.md |
| Unsaved editor / draft tab safety | references/instructions/unsaved-form-tabs.md |
| Windows subprocess stability (PIPE deadlock, SIGINT, tree kill) | references/instructions/windows-subprocess-cdp.md |
Done Criteria
- Verify the actual control route preserves OS foreground and user tab selection during the operation, not just before/after it; distinguish measured results from unverified behavior. Inspect saved captures for incomplete rendering.
- Report the work target and durable result; for repeat runs, report script reuse and compare elapsed time, tool calls, and retries without omitting verification. See the acceptance scenarios.
- Success is determined not only by toast but also by confirming via one of DOM / URL / API read / screenshot / status artifact
- On failure, report where it got stuck: modal / file chooser / iframe / CDP context
- What Playwright MCP writes directly under the working repo
.playwright-mcp/（console log / snapshot）が.gitignoreand any untracked files you created are checked and deleted withgit clean -nIf tracked,git rm -r --cached -- .playwright-mcp(file remains)