playwright-roll

Rolling Playwright Python

The goal of a roll is to move the driver pin in DRIVER_VERSION to a new release, port every public API change introduced upstream during that interval, and suppress the rest, so that ./scripts/update_api.sh and mypy playwright run clean and the test suite still passes.

The previous human-facing summary lives in ../../../ROLLING.md. This skill is the operational playbook — read it end to end before starting.

Mental model

The Python port is hand-written code in playwright/_impl/, plus a generator (scripts/codegen/generate_api.js) that:

parses the upstream docs ($PW_SRC_DIR/docs/src/api) with the upstream parser and filters them for Python,
applies the Python facts that the docs do not contain, from scripts/codegen/python_rules.js, and
emits the whole public API into playwright/{async,sync}_api/_generated.py: signatures, types, docstrings, event overloads, and the call into _impl/ for every member.

The generator never reads _impl/. A new member or option in the docs therefore shows up in _generated.py right away, as a call into _impl/ that does not exist yet. Each generated class types its _impl_obj, so mypy playwright lists every such gap as an error in _generated.py. (Temporary: while scripts/codegen/legacy_order.json exists, _impl_obj is not typed and mypy reports nothing. Find the gaps in the docs diff and with the tests instead.) Three resolutions:

PORT — the new API is intended for Python (no langs.only filter, or langs.only includes "python"). Implement it in _impl/ until mypy is clean.
MISMATCH — the shipped Python API differs from the docs (a callback signature uses unions, a kwarg uses a legacy name, etc.). The shipped implementation wins. Add an entry to scripts/codegen/python_rules.js with a comment explaining why, and propose the upstream docs change that would make the entry unnecessary.
N/A — the commit only touches docs, has * langs: js (or any other filter that excludes Python), is server-side, Electron-only, or was reverted later in the same release. No action; the generator already leaves it out.

The upstream documentation source of truth is docs/src/api/*.md in the playwright repo. Every ## method: / ## property: / ## event: / ### option: / ### param: block has an optional * langs: js (or js, python, etc.) filter. The Python doclint resolves these into langs fields on each member of api.json. An empty langs: {} means "all languages including Python" — implement it, don't suppress it.

The mistake the 1.59 roll made twice over: classifying things as "internal tooling, N/A for Python" based on the name of the API (Screencast, Debugger, pickLocator, clearConsoleMessages, artifactsDir, …). Almost all of those had empty langs: {} in api.json and were real Python APIs. Sounding tooling-y is not a langs filter. The langs field on the member in api.json is the only authoritative signal. When in doubt, dump it (see "Verifying classifications" below).

Process

1. Set up the env

CONTRIBUTING.md covers this. Notes from past rolls:

The repo requires Python 3.10+. If python3.10 isn't available, use python3 (3.12 is fine).
If python3-venv is missing system-wide, use uv venv env instead, then uv pip install --python env/bin/python --upgrade pip. Don't try to apt install — sudo is denied in the harness.
Always activate the venv before any pip, pytest, mypy, or pre-commit invocation.

2. Bump the driver pin, download it, and generate api.json

You need a nearby microsoft/playwright checkout for the docs walk and for codegen. Point PW_SRC_DIR at it and check out the release branch for the new version there (the branch has the release's docs plus any cherry-picks):

export PW_SRC_DIR=../playwright
git -C "$PW_SRC_DIR" fetch origin 'release-*:release-*'
git -C "$PW_SRC_DIR" checkout release-<new>      # e.g. release-1.62

Then bump the pins and assemble the driver:

# Edit DRIVER_VERSION (repo root): the playwright-core npm version for the new
# release, no "v" prefix, e.g.  1.62.0
python scripts/update_node_version.py     # refresh NODE_VERSION to the current LTS

source env/bin/activate
python -m build --wheel       # downloads playwright-core @ DRIVER_VERSION + Node.js, assembles the driver
playwright install chromium   # NOT --with-deps; sudo is denied

# The walk below inspects `langs` from api.json. Codegen does not need it.
node "$PW_SRC_DIR/utils/doclint/generateApiJson.js" > /tmp/api.json

The wheel build just downloads the playwright-core npm package at DRIVER_VERSION and the matching Node.js binary (no source build, no Node/npm/git toolchain), and unpacks the driver under playwright/driver/. The docs are the one piece not shipped in the bundle — ./scripts/update_api.sh reads them from $PW_SRC_DIR. /tmp/api.json is only a convenient dump of the same docs for the walk.

3. Identify the commit range

Use the nearby microsoft/playwright checkout at $PW_SRC_DIR (from step 2). Bring it up to date and ensure release branches/tags are present before walking the range:

git -C "$PW_SRC_DIR" fetch --tags
git -C "$PW_SRC_DIR" fetch origin 'release-*:release-*'

There is sometimes no vX.Y.0 tag for the latest release (the bots cut release branches first and tag later). Anchor on commits, not tags.

The diff range is "every commit on the new release branch since the previous release was cut". Anchor commits:

Previous release end: the chore: bump version to vX.Y.0-next commit on main. That commit is the first commit after the previous release (X.Y-1) was cut. Use its parent (<sha>~1) as the lower bound.
```
git -C "$PW_SRC_DIR" log --all --grep="bump version to v" --oneline | head
```
New release end: the tip of release-<new> (or the matching tag if it exists).

Save the commit list, oldest first, scoped to docs/src/api/:

git -C "$PW_SRC_DIR" log <prev-anchor>~1..release-<new> --oneline --reverse -- docs/src/api > /tmp/roll-<new>-commits.md

A normal roll yields 50–100 commits. If you see 0 or thousands, the range is wrong.

Format the file as a markdown checklist and add the standard preamble (status legend, where to look up api.json etc.) — see the file from the 1.58→1.59 roll for the template.

4. Walk the commit list

For each commit, in chronological order:

git -C "$PW_SRC_DIR" show <sha> -- docs/src/api/

Look for:

## (async )?method: / ## property: / ## event: additions or removals;
* langs: ... lines on those blocks;
### param: / ### option: additions or removals;
new class-X.md files (whole new classes — usually langs: js);
type changes in - returns: lines.

Classify and act.

Verifying classifications (do this before suppressing anything)

Before tagging anything as MISMATCH or N/A based on appearance, dump the actual langs from api.json:

import json
data = json.load(open("/tmp/api.json"))
classes = {c["name"]: c for c in data}
for cls_name in ["Page", "BrowserContext", "Screencast", "Debugger"]:
    cls = classes.get(cls_name)
    if not cls:
        continue
    print(f"\n{cls_name}: cls_langs={cls.get('langs', {})}")
    for m in cls["members"]:
        print(f"  {m['name']} kind={m.get('kind')} langs={m.get('langs', {})}")

For options/params nested inside an Object-typed arg, walk one level deeper:

for a in member.get("args", []):
    if a["name"] == "options":
        for prop in a.get("type", {}).get("properties", []):
            print(prop["name"], prop.get("langs", {}))

A few rules of thumb that catch most "actually a PORT" cases:

If the containing class has empty langs: {} and the member has empty langs: {}, it's for Python — implement it.
If the member is empty but a single option has langs: js, the method is for Python and you only skip that option (e.g. Screencast.start.size is langs: js while Screencast.start itself isn't).
If you're about to add three or more Method not implemented: entries for the same class, stop — you almost certainly need to implement the class.

PORT

Implement the change in playwright/_impl/<module>.py. Use the upstream JS implementation as a reference: $PW_SRC_DIR/packages/playwright-core/src/client/<module>.ts. Translate idioms:

Upstream JS	Python
`async foo(): Promise<X>`	`async def foo(self) -> X:`
`foo(): X` (sync getter, no args, no body)	`@property def foo(self) -> X:` (the generator makes argument-less sync members properties — see `isPythonProperty` in `scripts/codegen/generate_api.js` — and reads `self._impl_obj.foo` without calling it. mypy does not catch a plain method here; the test does.)
`await this._channel.foo({ a, b })`	`await self._channel.send("foo", None, locals_to_params(locals()))`
`(await this._channel.foo()).value`	`await self._channel.send("foo", None)` (Python's `send()` auto-unwraps single-key responses; only call `send_return_as_dict` when the protocol returns multiple keys.)
`(await this._channel.foo()).artifact` (multi-key, may be empty)	`result = await self._channel.send_return_as_dict("foo", None); (result or {}).get("artifact")` — `send_return_as_dict` returns `None` (not `{}`) when the protocol response carries no fields.
`try { ... } catch (e) { if (isTargetClosedError(e)) return; throw e; }`	`try: ...; except Exception as e: if is_target_closed_error(e): return; raise` (import from `playwright._impl._errors`)
Inline `[Object]` return like `{endpoint: string}`	A `TypedDict` in `playwright/_impl/_api_structures.py`. The generator names the struct from its docs `* alias:` (or a `structRenames` / `structNamesByPlace` entry in `python_rules.js`) and imports that name from `_api_structures.py`. See `RemoteAddr`, `BrowserBindResult`, `DebuggerPausedDetails`.
`binary` event/return field	The Python channel layer hands you a base64 string. Decode with `base64.b64decode(value)` before exposing as `bytes`. See `Screencast._dispatch_frame`.

When implementing a new ChannelOwner subclass (one constructed by the protocol with (parent, type, guid, initializer)):

Register it in playwright/_impl/_object_factory.py:create_remote_object — otherwise the guid resolves to DummyObject and downstream code breaks mysteriously.
Nothing to do in codegen: the generator emits every Python class in the docs. If the class does not live in _impl/_<snake_case name>.py, add it to implModules in scripts/codegen/python_rules.js.

When implementing a non-ChannelOwner wrapper class (a plain class that holds a Page/Context reference, like Screencast, Clock):

Set self._loop = parent._loop and self._dispatcher_fiber = parent._dispatcher_fiber in __init__. The generated AsyncBase/SyncBase wrappers read these; missing them gives AttributeError: 'X' object has no attribute '_loop' at first use.

When adding a new TypedDict in _api_structures.py:

Its name must match what the generator emits: the struct's * alias: in the docs, or a structRenames / structNamesByPlace entry in python_rules.js. The generator fails with "No Python name for the struct at …" until one exists.
Re-export it from both playwright/async_api/__init__.py and playwright/sync_api/__init__.py: assignment line plus an entry in __all__. Same pattern as ViewportSize, RemoteAddr.

If the new API was previously recorded in python_rules.js (for example in skipArgs), remove that entry when implementing it. The generator fails on entries that no longer match anything, so stale entries cannot pile up.

If a doc rename involves a positional parameter (no default, before any *), users almost certainly call it positionally — you can rename freely. The 1.59 BrowserType.connect.wsEndpoint → endpoint is the canonical example. Don't suppress this kind of rename; just rename in _impl/. Important corollary: when docs rename a param, the wire-protocol field usually also changed in packages/protocol/spec/*.yml and the server-side dispatcher in packages/playwright-core/src/server/dispatchers/*Dispatcher.ts. If so, you must also update the channel-send dict key (e.g. {"wsEndpoint": …} → {"endpoint": …}). An argNames entry for a renamed param is a code smell hiding a wire-protocol bug.

MISMATCH

A MISMATCH is a divergence between the docs and the shipped Python surface. The shipped implementation wins, but each entry is a debt: the real fix is an upstream docs change (a * langs: python declaration, an alias-python:, a Python type override), after which the entry goes away. Use it sparingly — most apparent mismatches turn out to be PORTs you skipped. Legitimate examples in the current python_rules.js:

Hidden internal kwargs (extraOptions: Browser.new_context(default_browser_type=)).
Callback signatures where Python explicitly unions one-arg and two-arg variants but the docs document only the canonical form (typeOverrides: Page.route.handler, WebSocketRoute.onClose.handler).
Positional parameters that the docs declare as options (positional: select_option(value, ...)).

Add an entry to the right table in scripts/codegen/python_rules.js, with a one-line // comment above it explaining why. Keys use docs names (Class.member.arg, camelCase). Example:

  typeOverrides: {
    // One vs two arguments in the callback, Python explicitly unions.
    'Page.route.handler': '[function]([Route])|[function]([Route], [Request])',
  },

The generator fails with "the entry … no longer matches anything; remove it" when an upstream change makes an entry unnecessary. Delete that entry.

Do not suppress these — they're PORTs in disguise:

"Internal tooling" classes/methods whose langs field is empty (Screencast.*, Debugger.*, Page.pick_locator, BrowserContext.debugger, Browser.bind/unbind, Page.{clear,console}_*). The 1.59 roll suppressed all of these initially, then had to walk every one back. Verify langs first.
A renamed positional parameter (mypy: unexpected keyword argument newName for the impl method). Just rename in _impl/ and update the channel-send dict key.
A struct return the generator cannot name ("No Python name for the struct at …"). Add a TypedDict and name it — don't type it Dict.

N/A

Common N/A flavors:

Whole new class with langs: js (Disposable, Inspector/Screencast, Debugger, Overlay).
Members with langs: js (most "tooling" / MCP / agentic features).
Doc-only edits (typo fixes, "Improve not property sections", etc.).
Reverts that cancel an earlier add in the same range (always check the rest of the range before porting something that gets reverted).
Java/C# langs: blocks.
Electron-only changes (docs/src/api/class-electron.md).

Tick the box in /tmp/roll-<new>-commits.md with one line: [x] <sha> <subject> — <classification>: <one-liner>.

5. Regenerate

PW_SRC_DIR=../playwright ./scripts/update_api.sh   # PW_SRC_DIR already exported in step 2

The script, for sync and then async:

runs node scripts/codegen/generate_api.js --flavor <flavor> (reads the docs from $PW_SRC_DIR) straight into playwright/<flavor>_api/_generated.py,
invokes pre-commit run --files on the generated file.

Then run mypy playwright and work through the errors in _generated.py — each one is a call into _impl/ that the implementation does not support yet.

To see exactly what a regeneration changed in the public API, compare against the committed file:

git show HEAD:playwright/async_api/_generated.py > /tmp/old.py
python scripts/codegen/compare_api.py /tmp/old.py playwright/async_api/_generated.py --show all

CI no longer verifies that _generated.py is in sync (the Lint job dropped the "Verify generated API is up to date" step so it needn't check out upstream). So regenerating here and committing the result is on you — don't skip it.

Failure modes and fixes:

Symptom	Cause	Fix
mypy: `"XImpl" has no attribute "y"` in `_generated.py`	The docs have a new member `X.y`; `_impl/` does not.	PORT it.
mypy: `Unexpected keyword argument "z" for "y" of "XImpl"`	New parameter or option on an existing method.	Add the kwarg (docs camelCase name) in `_impl/`. If Python must not have it, `skipArgs`.
mypy: incompatible argument type in `_generated.py`	The docs type changed, or `_impl/` declares a narrower type.	Match the type in `_impl/`, or record a `typeOverrides` entry for a justified divergence.
generator: `No Python name for the struct at X.y.z`	A new inline `[Object]` with no `* alias:` in the docs.	Add the TypedDict and name it: `alias-python` upstream, or `structNamesByPlace` / `structRenames`.
generator: `Unknown docs type "T" at X.y`	A docs type the type table in `generate_api.js` (`toPy`) does not know.	Add it to `toPy`.
generator: `required parameter z follows an optional one`	The docs order params so that Python would need a default before a non-default.	Add a `positional` entry with the Python order.
generator: `the entry "…" no longer matches anything; remove it`	An upstream change made a rules entry unnecessary (or renamed what it points at).	Delete it, or re-key it to the new docs name.
generator: `legacy_order.json: … has no …; remove it`	A member or parameter in the temporary legacy order was removed or renamed upstream.	Delete that name from `scripts/codegen/legacy_order.json`. New members need no entry; they go last.
`pyright … reportInconsistentOverload` (single-event class)	A class gained its first event, so there is one `Literal[…]` overload + impl, which pyright wants ≥2 of.	The generator already handles this — `emitEvents` adds a second `@typing.overload` with `event: str` when a class has exactly one event.
`pre-commit` keeps reformatting `_generated.py` on each run	First run after regen always reformats once; rerun until idle.	`pre-commit run --all-files` to settle.
A kwarg in `_impl/` that the docs don't mention (e.g. legacy name from a doc rename)	Nothing fails: the generator simply never passes it.	If the param is positional with no default, just rename it in `_impl/`. Check `packages/protocol/spec/*.yml` and the server dispatcher — if the wire field renamed too, also update the channel-send dict key. Only use `extraOptions` for genuinely hidden internal kwargs (`default_browser_type`).
`AttributeError: 'X' object has no attribute '_loop'` (or `_dispatcher_fiber`) at first use of a new wrapper class	The non-ChannelOwner wrapper isn't initializing the fields the generated `AsyncBase`/`SyncBase` reads.	In the wrapper's `__init__`, set `self._loop = parent._loop` and `self._dispatcher_fiber = parent._dispatcher_fiber`.
`'NoneType' object has no attribute 'get'` after `send_return_as_dict`	Method's protocol response carries no fields and `send_return_as_dict` returned `None`.	`(result or {}).get(...)`.
Frame/buffer payload arrives as a `str` instead of `bytes`	Protocol `binary` fields cross the wire as base64.	`base64.b64decode(value)` in the impl before exposing.

After the script settles, run pre-commit run --all-files once more to confirm everything is idle.

6. Add tests

For each PORT, add one async test and a matching sync test. Conventions:

Tests go in the existing topic file (test_page_network_request.py, test_browsercontext.py, test_dialog.py, …) — don't create new files unless there's no obvious home.
Use from playwright.async_api import …, not from playwright._impl._page import Page (the impl class doesn't have the public wrappers like expect_console_message).
For event-info objects, await message_info.value (it's an async property).
Don't write tests that hang the page (e.g. page.evaluate(... fetch slow ...) followed by page.close() from a fixture) — the request task gets a TargetClosedError. Use page.on("event", handler) to capture state at event time instead.
playwright install chromium (no --with-deps) is sufficient for the test suite under sandbox.

7. Update existing high-touch artifacts

DRIVER_VERSION and NODE_VERSION: already done in step 2.
README.md: gets the chromium/firefox/webkit version table updated automatically by scripts/update_versions.py (called from update_api.sh). Don't edit by hand.
The "Backport changes" tracking issue on GitHub (filed by microsoft-playwright-automation) is the intent tracker, but it's frequently out of sync with what's actually been ported. Treat it as a starting point, not the source of truth — the docs/src/api/ commit walk is authoritative.

8. Final verification

pre-commit run --all-files
mypy playwright                 # 2 pre-existing errors in _json_pipe.py and _artifact.py are unrelated
pytest --browser chromium tests/async/<files-you-touched> tests/sync/<files-you-touched>

Then a smoke regression on a few neighboring suites (tests/async/test_browser*.py, test_cdp_session.py, test_tracing.py, test_dialog.py, test_page_*.py) to make sure nothing inherent to the port shifted.

Reference: `python_rules.js` tables

All keys use docs names: Class.member or Class.member.arg, with the upstream PascalCase class and camelCase member/arg names. * matches one name segment.

Table	Use
`extraClasses`	A class the docs mark for other languages only, but Python has (`Disposable`).
`handWrittenClasses`	Classes in the Python docs that are written by hand (`Error`, `TimeoutError`, `FormData`).
`contextManagerClasses`, `tracebackHideClasses`	Base-class and `__tracebackhide__` choices.
`extraMembers`, `extraOptions`	Python-only members and hidden options; generated without docstrings.
`skipArgs`	Docs arguments Python does not have.
`argNames`	Docs arguments whose Python name differs.
`typeOverrides`	Python types that differ from the docs (docs type syntax; a function type with a return type must be last in a union). A member place overrides the return type.
`positional`	Methods whose positional parameters differ from "params positional, options keyword-only".
`positionalOptions`	Option names that are positional everywhere (`predicate`).
`structRenames`, `structNamesByPlace`	TypedDict names.
`implModules`	`_impl/` module of classes not in `_<snake_case name>.py`.

Tips & gotchas

langs.only is your filter — and the only filter. Don't classify by name (Screencast, Debugger, pickLocator) or by intuition ("looks like internal tooling"). Always check langs in api.json. The 1.59 roll cost two extra audit passes by trusting names over langs.
Audit your own classifications a second time. After the first walk through the commit range, before opening the PR, re-read every entry in python_rules.js you added during this roll and ask "is this divergence justified, or did I skip a port?" Run the langs-dump snippet on each suspicious entry. The 1.59 roll's first PR had ~20 wrong suppressions; the second pass cut them to 0.
A cluster of rules on the same class is a smell. If you're about to add five entries for Foo.*, you're almost certainly looking at a class that needs to be implemented. Implement the whole thing once and the entries disappear.
Watch for revert pairs in the same range. 1.59 added and reverted Browser.isRemote (#39613 / #39620) inside the same release. Walking chronologically lets you skip the add when you see the revert later.
Watch for rename-revert pairs. 1.59 had Locator.normalize → Locator.toCode (#39648) → Locator.normalize (#39754). Final state wins; only port the last.
Doc renames almost always include a wire-protocol rename. Whenever you see ### param: X.y.oldName → ### param: X.y.newName in a doc commit, also git -C "$PW_SRC_DIR" show <sha> -- packages/protocol/spec/ and the corresponding *Dispatcher.ts file. If the wire field changed too, the channel-send dict key in _impl/ must change. Suppressing the doc-side mismatch is hiding a real bug — the previous Python code is silently sending an unknown field that the new server ignores.
TypedDicts beat Dict[str, X] for any structured return. When the docs describe a return as [Object] with named fields, define a TypedDict in _api_structures.py, name it for the generator, and re-export it from both public __init__.py files. Zero runtime cost (it's still a dict), and the docstrings expand its fields from the docs.
Positional renames are free. A param with no default before any * separator is positional-or-keyword in Python, but realistic call sites pass it positionally. Renaming such a param doesn't break callers.
The "Backport changes" GitHub issue can be misleading. In the 1.59 roll its checkboxes were all marked [x] with annotations like "✅ IMPLEMENTED", but several of those features had not actually been merged into the Python port. Trust the docs/src/api/ walk over the issue.
The docs parser has quirks. In a type expression, everything after a function's : is its return type, so [function]([A]):[any]|[B] is one function returning any|B. Put a function with a return type last in a union. The parser also drops the second argument of a function type when the first is a union (see the WebSocketRoute.onClose.handler override, which builds the type by hand). And when a ### param: header name differs from the backticked name on its type line, the header name is the docs name and the type-line name becomes the alias (see argNames).
Don't edit _generated.py to fix lint or typing. Fix _impl/, scripts/codegen/generate_api.js, or python_rules.js instead. Hand-editing the generated file is reverted on the next regen.
/tmp/roll-<new>-commits.md is a working artifact, not a deliverable. Don't commit it. The commit message and PR description are where the audit summary belongs.