When to use
- About to do a task that touches UI outside the browser and not sure which tool to call.
- A tool just failed (AX blind, localhost blocked, dead ref) and you need a justified fallback.
Steps
- Answer two questions, get the answer immediately: is the target INSIDE the browser or outside? Is the machine running macOS or not?
| task | tool | why |
|---|---|---|
| Website / web app, need DOM control | /playwright-verify |
cheapest, stable selectors, can run file:// and localhost |
| Multi-step website, keep session | /playwright-verify mode CDP, or /dym-browser-use repl |
tab and cookie survive between turns |
| Website in user's real Chrome | claude-in-chrome |
uses the user's actual profile/login; blocks localhost and file:// |
| macOS desktop app native (System Settings, Finder, Numbers, Notes) | agent-desktop | durable refs, skeleton, actionability, branchable error codes |
| Desktop app on Linux/Windows | orca computer |
agent-desktop only supports darwin |
| UI of the Orca app itself, Orca worktree/terminal | orca-cli / orca computer |
same runtime, no extra binary needed |
| Electron app without AX enabled (Teams, some Slack builds) | launch --cdp + CDP client |
both AX tools are blind at the "Web content" boundary |
- If choosing agent-desktop →
/dym-agent-desktop installthen/dym-agent-desktop drive. - If you need a receipt →
/dym-agent-desktop session.
Measured comparison (2026-09-09, macOS 24.6 arm64, agent-desktop 0.8.5)
| axis | orca computer |
agent-desktop |
|---|---|---|
| Platform | macOS + Linux (orca-ide) |
darwin only (SUPPORTED_PLATFORMS = ['darwin']; crate windows/linux 76-line stub) |
| Install cost | 0 — bundled with the Orca app | +3.0M binary + 498K helper via npm |
| Element identification | element-index, lives within a single return turn, its own docs say "short-lived, go stale" | @<snapshot_id>:eN, refmap written to disk → resolvable in another process, minutes later |
| Dead ref | no dedicated error code — wrong index points at the wrong element | STALE_REF / SNAPSHOT_NOT_FOUND + recovery.strategy |
| Ambiguous window | picks a window itself, no warning (Teams 2 windows → returns w-9249) | AMBIGUOUS_TARGET + lists all candidates |
| Token cost, dense app | System Settings: treeText 3313 characters / 117 lines, whole tree in one shot | full -i --compact 24185 B / 168 ref; skeleton 4630 B / 27 ref then drill |
| Command surface | ~15 commands | 55 working commands / 59 names (surfaces, notifications, clipboard, wait, batch, session, trace) |
| Headless | undeclared — actions go through the physical path | headless by default, --headed only steals focus/moves the mouse |
| Evidence | none | trace JSONL + trace export to HTML |
| Multiple agents | no such concept | shared session + AGENT_DESKTOP_AGENT_ID + cursor lease |
Rules
- Don't read "agent-desktop is cheaper in tokens" from the table above. At full depth it is MORE expensive
orca computerby about 5 times per element (JSON vs indented treeText). What it saves is the number of turns — skeleton + drill + ref don't die, so fewer rounds of re-snapshotting from scratch. - Work that is one turn, one sparse app, read and done →
orca computerstill wins: 0 install, 1 command. - Work that is multi-step, has mutations, needs verification and needs receipts → agent-desktop clearly wins.
- No tool here can replace another on fundamentals: on Linux, agent-desktop is the number 0.
- On the web, don't use either — the browser's AX is the longest path to a button.