SuperAPI AI Directory
Partner中文AI newsFavorites
Skillsreverse-browser-automation

reverse-browser-automation

Agents & models

View repository

Updated 10-20 02:29

About this skill

Automation (Desktop & Browser Automation)

When to Use

  • Analysis requires scripted interaction with a browser or desktop application.
  • Collecting reproducible UI evidence (screenshots, network traces) during an assessment.

Scope

Use this skill when the task falls into the following scenarios:

Browser scenarios (Playwright / agent-browser)

  • Open a web page and interact with page elements (click, fill in forms, submit)
  • Scrape page content or take screenshots
  • Automate login flows
  • Interact with web pages during penetration testing (submit payloads, trigger XSS)
  • Automated handling of CAPTCHA pages
  • Batch form submission

Desktop application scenarios (OpenReverse)

  • Operate Windows desktop applications (IDA Pro, x64dbg, Wireshark, etc.)
  • Requires vision-driven interaction (CUA mode)
  • Requires structured UI operations (UIA mode)
  • Network traffic observation for desktop applications (built-in mitmproxy)
  • GUI operations for automated reverse-engineering tools
  • Black-box testing of desktop software

Division of labor with other tools

Scenario What to use
Operate web pages (inside the browser) Playwright / agent-browser
Operate desktop applications (Windows GUI) OpenReverse
Packet capture analysis, HTTP request capture anything-analyzer or OpenReverse network lane
JS breakpoints, Hooks, CDP debugging jshookmcp
Locate signature algorithms, reproduce with environment shimming js-reverse

Simple decision:

  • Target is a web page → Playwright
  • Target is a Windows desktop application → OpenReverse
  • Both are needed → use them in combination

Part 1: Browser automation (Playwright / agent-browser)

Core workflow

# 1. 打开页面
agent-browser open <url>

# 2. 获取可交互元素（返回 @e1, @e2... 引用）
agent-browser snapshot -i

# 3. 用引用操作元素
agent-browser click @e1
agent-browser fill @e2 "text"

# 4. 完成后关闭
agent-browser close

Command reference

# 导航
agent-browser open <url>
agent-browser close

# 页面快照
agent-browser snapshot        # 完整无障碍树
agent-browser snapshot -i     # 仅可交互元素（推荐）

# 交互操作
agent-browser click @e1
agent-browser fill @e2 "text"
agent-browser type @e2 "text"
agent-browser press Enter
agent-browser scroll down 500

# 获取信息
agent-browser get text @e1
agent-browser get title
agent-browser get url

# 等待
agent-browser wait @e1
agent-browser wait 2000
agent-browser wait --load networkidle

Notes

  • Must execute agent-browser close, otherwise processes will leak
  • Take a snapshot before operating; do not guess element references
  • After submitting a form, use wait --load networkidle and other methods to wait for the page to stabilize

Part 2: Desktop application automation (OpenReverse)

Overview

OpenReverse is a desktop interaction and evidence collection framework for AI Agents, supporting:

  • UIA mode: Windows UI Automation, structured desktop control operations
  • CUA mode: vision-driven interaction (Computer Use Agent), suitable for complex GUIs
  • Network observation: built-in mitmproxy proxy + local capture

Interaction mode selection

Mode Suitable scenarios Underlying mechanism
UIA The target application has standard Windows controls (buttons, text boxes, lists) Windows UI Automation API
CUA The target application's UI is complex or has non-standard controls (IDA's disassembly view, custom-rendered interfaces) Visual recognition + mouse and keyboard

Network observation mode

Mode Suitable scenarios
Proxy Lane The target application can be configured with a proxy (recommended)
Local Lane The target application cannot go through a proxy and requires local capture

Installation and configuration

# 1. Clone 项目
git clone https://github.com/zhexulong/openreverse.git
cd openreverse

# 2. 安装依赖
npm install

# 3. 接入 Agent 宿主（Claude Code / Codex / Zed）
npm run init:agents -- --target=all /path/to/project

# 4. 安装 CUA runtime（如果需要视觉驱动模式）
npm run install:cua-runtime
npm run doctor:cua-runtime

# 5. 安装网络观察依赖（如果需要抓包）
npm run install:mitmproxy
npm run doctor:network

Common combinations

Requirement Configuration
Only operate desktop applications UIA or CUA, without connecting the network lane
Operate desktop applications + packet capture UIA/CUA + proxy lane
Operate desktop applications + local capture UIA/CUA + local lane

Reverse Engineering Scenario Examples

场景：自动化操作 IDA Pro 进行批量分析

1. 用 OpenReverse CUA 模式打开 IDA Pro
2. 自动加载目标二进制
3. 等待分析完成
4. 通过 UI 操作导出函数列表
5. 同时用 network lane 观察 IDA 的网络行为（如 Lumina 请求）

场景：自动化操作 x64dbg 调试

1. 用 OpenReverse UIA 模式启动 x64dbg
2. 加载目标程序
3. 设置断点
4. 运行并观察寄存器/内存变化
5. 截图保存证据

On-Demand Bootstrap

Automation Capability Boundaries

Tool Auto-installable Installation Method Description
Playwright ✓ npm + npx playwright install Browser automation engine
agent-browser CLI ✓ npm install -g agent-browser Browser operation CLI
Node.js ✓ winget Prerequisite dependencies
OpenReverse ✗ Manual clone + npm install Experimental stage, heavy dependencies
mitmproxy ✗ Manual installation OpenReverse network observation dependency

Bootstrap Trigger

  • Browser operation missing Playwright → automatic bootstrap
  • Desktop operation requires OpenReverse → guide the user to install manually (provide complete steps)

OpenReverse Manual Installation Guide

If the AI detects that desktop application automation is needed but OpenReverse is not installed:

⚠️ **需要 OpenReverse 进行桌面应用自动化**

**安装步骤**：
1. `git clone https://github.com/zhexulong/openreverse.git`
2. `cd openreverse && npm install`
3. `npm run init:agents -- --target=all <你的项目路径>`
4. 如需视觉模式：`npm run install:cua-runtime`
5. 如需网络观察：`npm run install:mitmproxy`

**验证**：`npm run doctor:cua-runtime` 和 `npm run doctor:network`

Routing Context

Upstream Entry: skills/SKILL.md(master control),routing.md Applicable Scenarios: Any task that requires automated operation of a browser or desktop application Downstream Exit:

  • Captured requests need analysis → anything-analyzer or js-reverse
  • JS debugging/Hook needed → jshookmcp
  • Signature algorithm restoration needed → js-reverse
  • Desktop application is a reverse engineering tool → ida-reverse/

Peer Related Modules: js-reverse(may need to analyze JS after browser operations),ida-reverse(OpenReverse can automate IDA GUI operations)

Task Completion Self-Check (MUST pass before claiming completion)

  • Did I execute each step in the workflow (rather than only reading)?
  • Did I base it on tool-index using real tool paths?
  • Did I produce reproducible evidence (commands/scripts/screenshots/reports)?
  • Did I complete and write back the Checklist items required by RULES?

Limitations

  • UI selectors break on application updates; scripts need maintenance.
  • Windows desktop automation requires an OS with the matching accessibility stack.

Adapted from zhaoxuya520/reverse-skill (MIT).