SuperAPI AI Directory
Partner中文AI newsFavorites
Skillspeach-e2e-browse

peach-e2e-browse

Agents & models

View repository

Updated 10-15 21:39

About this skill

AI browser navigation

agent-browser(default) + playwright-clicontrols Chrome Beta via (fallback). The AI takes over the browser in which a human has completed login in the Chrome Beta fixed profile and navigates it.

Can run standalone: even without the e2e/ folder or ./e2e.sh setup, if only agent-browser is installed, the core functions (navigation/verification/checking) work.

Enforced gate

The following 5 items are not recommendations but enforced rules.

  1. For launching Chrome Beta, cd e2e && ./e2e.sh chromeis used first. If direct execution is needed, be sure to include --user-data-dir=$HOME/.chrome-beta-e2e-profile.
  2. agent-browser connect 9222 After that, be sure to run agent-browser tab listfirst.
  3. Do not perform any operation until the user explicitly specifies the tab number.
    • Prohibited: open, tab, click, fill, press, eval
  4. For tasks where Google login, OAuth, admin console, payment, or maintaining an existing profile session is important, perform analysis only and proceed with execution after user confirmation.
  5. agent-browserIf malfunctions, OS-level workarounds (open -a, launching another browser, using another profile path) are prohibited. Immediately report the status and get user confirmation.

When reaching a page that requires authentication, the AI does not attempt login/secondary authentication itself. In Chrome Beta's $HOME/.chrome-beta-e2e-profile, guide the user to complete login or authentication themselves, then the AI takes over the same profile session.

Chrome Beta launch invariant rules

When launching Chrome Beta in CDP mode, the fixed profile option is mandatory. A launch missing the profile option is considered a session-maintenance failure.

  • Allowed: cd e2e && ./e2e.sh chrome
  • Required options for direct execution: --remote-debugging-port=9222, --remote-allow-origins=*, --user-data-dir=$HOME/.chrome-beta-e2e-profile, --disable-extensions
  • macOS direct execution standard: /usr/bin/open -na "Google Chrome Beta" --args ...
  • Prohibited: open -a "Google Chrome Beta" standalone execution, --args without open execution, --user-data-dir launching Chrome Beta without , arbitrarily using another profile path, bypassing via default Chrome or another browser

Tool role division

Purpose Tool Reason
Navigation/verification/checking agent-browser eval 6.6x faster, 2.3x token savings
Scenario execution playwright-cli (./e2e.sh run) Scenario infrastructure based on lib/connect.js
fallback playwright-cli (direct call) Features unsupported by agent-browser, such as iframe

Decision tree (check first, highest priority)

웹사이트 도착
│
├─ connect 9222 → tab list → 사용자 탭 번호 확인
│  ├─ 탭 미확인
│  │  → 상태만 보고하고 대기
│  ├─ Google/OAuth/관리자/결제/기존 세션 유지 작업
│  │  → 탭 번호 확인 후에도 분석만 수행, 실행은 사용자 승인 후
│  └─ 일반 탐색 작업
│     → 아래 흐름 진행
│
├─ eval "document.title + ' | ' + location.href" 로 현재 상태 파악
│
├─ 정상 텍스트 반환 (일반 HTML / SPA)
│  ├─ 데이터 읽기 → eval 한 줄로 해결
│  ├─ 셀렉터를 알고 있다 → eval로 click/value 설정
│  │  ⚠️ eval click()은 isTrusted:false — 보안 폼에서 안 될 수 있음
│  │  → 안 되면 snapshot -i -c -s "form" → click ref
│  ├─ 셀렉터를 모른다
│  │  ├─ find role button --name "버튼명" (snapshot 없이 탐색)
│  │  └─ snapshot -i -c -s "관심영역" → ref 확보 → click/fill
│  └─ SPA 로딩 대기 필요
│     → wait --load networkidle 또는 eval "!!querySelector('타겟')" 체크
│  └─ SPA 입력 후 버튼이 [disabled] 유지
│     → 이벤트 미발행 의심 (Angular ngModel / React controlled input)
│     → eval "el.dispatchEvent(new Event('input', {bubbles:true}))"
│     → 여전히 [disabled] → change 이벤트도 추가 발행
│     → 여전히 [disabled] → CDP Input.insertText (자동화 스크립트 필요)
│     (references/SPA-프레임워크-입력패턴.md §1 참조)
│  └─ 파일 업로드 필요 (input[type=file])
│     → 기본값: CDP 인터셉트 방식 (처음 시도하는 사이트는 항상 이 방법)
│        인터셉트 스크립트 백그라운드 실행 (TAB_TARGET_ID로 탭 특정)
│        → agent-browser click (isTrusted:true 필수 — eval click은 무시됨)
│        → Page.fileChooserOpened 이벤트 → setFileInputFiles(backendNodeId)
│     ⚠️ 직접 주입(DOM.setFileInputFiles)은 OS 다이얼로그가 안 뜨는 사이트임이
│        검증된 경우에만 허용. 다이얼로그가 열리면 CDP/Escape로 닫을 수 없어 세션 막힘.
│     (references/SPA-프레임워크-입력패턴.md §3 스크립트 템플릿 참조)
│
├─ "로딩 중입니다" or 빈 값 → Flutter/Canvas 의심
│  ├─ eval "!!querySelector('flt-semantics-placeholder')" → Flutter 확정
│  ├─ accessibility 활성화
│  │  ├─ eval "querySelector('flt-semantics-placeholder')?.click()"
│  │  └─ 안 되면 snapshot -i -c → click e2 (flutter-view)
│  ├─ 활성화 후 → snapshot -i -c로 ref 확보 → 메뉴 조작
│  ├─ ⚠️ 페이지 이동마다 accessibility 재활성화 필요
│  └─ **우회 전략 (권장)**: dart.js에서 API method 추출 → fetch 직접 호출
│     (references/Flutter-웹앱-패턴.md 참조)
│
├─ iframe 내부 접근
│  → playwright-cli fallback (references/iframe-모달-패턴.md 참조)
│
└─ 세션 끊김 (CDP error: Session not found)
   → connect 9222 → tab list → tab N → eval "document.title"

Key: check the tab first, eval next. snapshot is the last resort. For Flutter, direct API calls are the strongest.

Workflow

1. 환경 확인 (인라인 체크 — e2e/ 폴더 불필요)
2. CDP 연결 (agent-browser connect 9222)
3. tab list → 사용자에게 탭 목록 보여주고 작업 탭 확인
4. 선택된 탭에서 작업 (eval/click/snapshot)
5. 결과를 사용자에게 보고

Connect after reporting results

If the navigation results are worth repeated verification, organize the following at the end of the report.

  • Scenario feasibility
  • Reusable URLs, selectors, input values, verification points
  • Recommended next step: /peach-e2e-scenario create ...

If the task is just a simple check, do not create a scenario file.

Stop conditions

  • Tab number unconfirmed
  • Task where keeping the Chrome Beta fixed profile is essential
  • Sensitive session tasks such as login/OAuth/admin console/payment
  • agent-browser connect, tab list, tab newstate where is an abnormal response

Under the above conditions, do not manipulate anything; report only the current state and wait for user instructions.

Step 1: Environment check (standalone execution)

Check directly without an e2e/ folder or setup script:

# a) agent-browser 설치 여부
command -v agent-browser && echo "✅ agent-browser 설치됨" || echo "❌ agent-browser 미설치 → npm i -g agent-browser"

# b) Chrome Beta CDP 연결 여부
curl -s http://127.0.0.1:9222/json/version && echo "✅ CDP 연결됨" || echo "❌ CDP 미연결"

If CDP is not connected, first attempt automatic recovery.

  1. e2e/ If the folder exists, cd e2e && ./e2e.sh chrome & after running sleep 4, cd e2e && ./e2e.sh statusrecheck with .
  2. e2e/ If the folder does not exist, directly run the OS-specific Chrome Beta CDP launch command below, then recheck.
  3. If it is still not connected after rechecking, then guide the user to run it manually.

Run Chrome Beta CDP directly (when CDP is not connected)

⚠️ Profile path fixing rule: --user-data-dir must be $HOME/.chrome-beta-e2e-profile fixed. Under no circumstances change it to another path. Even if an error occurs, do not change the path. If you keep authentication such as a Google account, OAuth, or corporate SSO in this profile in advance, the AI can later take over the same session and work.

macOS:

# --disable-extensions: 확장 프로그램이 웹 콘텐츠에 다크모드를 강제 적용하는 문제 방지
/usr/bin/open -na "Google Chrome Beta" --args \
  --remote-debugging-port=9222 \
  "--remote-allow-origins=*" \
  "--user-data-dir=$HOME/.chrome-beta-e2e-profile" \
  --disable-extensions \
  --no-first-run \
  --no-default-browser-check
sleep 5
curl -s "http://127.0.0.1:9222/json/version"

Windows (Git Bash):

# --disable-extensions: 확장 프로그램이 웹 콘텐츠에 다크모드를 강제 적용하는 문제 방지
"/c/Program Files/Google/Chrome Beta/Application/chrome.exe" \
  --remote-debugging-port=9222 \
  --remote-allow-origins=* \
  --user-data-dir="$(cygpath -w "$HOME/.chrome-beta-e2e-profile")" \
  --disable-extensions &

If an e2e/ folder exists, cd e2e && ./e2e.sh chromecan also be run with .

Step 2: CDP connection

agent-browser connect 9222

Once connected, the session is maintained. No need to reconnect for every command.

Before manual verification, checking the resident CDP session is mandatory agent-browser connect 9222 When runs, a resident daemon is created. If this daemon is attached, the site's native alert/confirm/promptmay close automatically. If the task requires the user to check the dialog directly, first check the connection status.

lsof -iTCP:9222 -sTCP:ESTABLISHED -n -P   # 연결 확인
ps -p <PID> -o pid,ppid,command=            # daemon 식별
kill <PID>                                   # daemon 종료

Details: references/native-dialog-주의사항.md see

Step 3: Check tabs → ask the user to choose

agent-browser tab list

Show the tab list to the user and "Which tab number should I work in?" confirm. Output example:

  [0] Daum - https://www.daum.net/
  [1]  - chrome://webui-toolbar.top-chrome/
→ [2] NAVER - https://www.naver.com/
  [3] Google - https://www.google.com/

→ The indicator is agent-browser internal focus, not Chrome UI focus. CDP cannot know which tab the user is viewing in Chrome. Therefore, before starting work, you must confirm the tab number with the user.

If the user specifies a tab number, agent-browser tab Nswitch with and proceed.

Before receiving the tab number, stop here. open, tab, click, fill, press, evalDo not continue executing . For tasks where maintaining the session is essential, such as Google/OAuth/admin console, even if you receive the tab number, if there is no additional execution approval, perform analysis only. agent-browser tab N Immediately after , always eval "document.title + ' | ' + location.href"recheck with whether it is the expected tab. If different, tab listprint again and reselect.

Step 4: Manipulation

In the selected tab: eval → judge → additional eval/click.

Step 5: Report results

Summarize the eval results and page state as text and report to the user.

Tab rules

Work in the current tab. Switch tabs only when the user explicitly instructs it.

  • If the user specifies a URL → in the current tab agent-browser open "URL"
  • "Go to tab N" → agent-browser tab N
  • "Open in a new tab" → agent-browser tab new "URL"
  • The AI must not arbitrarily switch tabs or open new tabs

Until the tab number is confirmed, do not execute any of the above commands. In sensitive session work, even after confirming the tab number, perform analysis only until the user approves execution.

open URL --new-tab Do not use! It overwrites the existing tab. For a new tab, always tab new "URL" use.

Manipulation commands

Page navigation

agent-browser open "https://대상URL"

JavaScript execution (default manipulation method)

# 값 읽기
agent-browser eval "document.title"
agent-browser eval "document.querySelector('#field').value"
agent-browser eval "document.querySelectorAll('tr').length"

# 조건부 읽기
agent-browser eval "document.querySelector('.cls') ? document.querySelector('.cls').innerText.trim() : '없음'"

# 목록 추출 (JSON.stringify 패턴)
agent-browser eval "JSON.stringify(Array.from(document.querySelectorAll('li')).map(function(el){return el.innerText}))"

# 클릭
agent-browser eval "document.querySelector('a.link').click()"

# 값 입력
agent-browser eval "document.querySelector('#keyword').value = '검색어'"

eval is for simple expressions only. IIFE (function(){...})() Do not use -- serialization errors occur. Split multiple actions into separate evals and run them individually.

Element search (snapshot -- only when unavoidable)

# 인터랙티브 요소만 + 컴팩트 출력 (필수 옵션)
agent-browser snapshot -i -c

# CSS 범위 제한 (더 절약)
agent-browser snapshot -i -c -s "table"

Full snapshot prohibited. Token comparison:

  • Full snapshot: ~65,700 tokens
  • snapshot -i -c: ~9,800 tokens
  • eval: 1460 tokens

After snapshot, manipulate by ref:

agent-browser click e10
agent-browser fill e37 "검색어"
agent-browser press Enter

ref becomes invalid when the DOM changes. After clicking/navigating, always snapshot again.

Click / input / keyboard (ref-based)

agent-browser click e10
agent-browser fill e37 "검색어"
agent-browser press Enter

Screenshot

agent-browser screenshot          # 터미널 출력
agent-browser screenshot result.png  # 파일 저장 (토큰 미소비)

fallback: playwright-cli (iframe/modal)

If agent-browser cannot support it, switch to playwright-cli.

When switching is needed

  • Accessing elements inside an iframe (jQuery UI Dialog + iframe modal)
  • When only the iframe tag appears in the agent-browser snapshot and the inside is empty

Pre-check (e2e/ folder not required)

# playwright-cli 설치 확인
command -v playwright-cli || echo "❌ 미설치 → npm i -g @playwright/cli"

# config 파일 확인 — 없으면 생성
if [ ! -f "$HOME/.playwright/cli.config.json" ]; then
  mkdir -p "$HOME/.playwright"
  echo '{"browser":{"cdpEndpoint":"http://localhost:9222","isolated":false}}' > "$HOME/.playwright/cli.config.json"
  echo "✅ playwright-cli config 생성됨"
fi

isolated: false Required — default value trueIf so, a new browser opens even when connecting to CDP.

Direct call method

In the e2e/ folder, pwc.sh Call directly without the wrapper:

# playwright-cli 세션 오픈 (1회)
playwright-cli --config="$HOME/.playwright/cli.config.json" open

# 탭 전환
playwright-cli --config="$HOME/.playwright/cli.config.json" tab-select N

# iframe 내부 읽기
playwright-cli --config="$HOME/.playwright/cli.config.json" eval "document.querySelector('iframe[src*=대상]').contentDocument.querySelector('#element').innerText"

# iframe 내부 입력 + 클릭 -- 별도 eval로 분리
playwright-cli --config="$HOME/.playwright/cli.config.json" eval "document.querySelector('iframe[src*=대상]').contentDocument.querySelector('#keyword').value = '검색어'"
playwright-cli --config="$HOME/.playwright/cli.config.json" eval "document.querySelector('iframe[src*=대상]').contentDocument.querySelector('input[type=submit]').click()"

If there is an e2e/ folder, ./e2e/pwc.sh using the wrapper is the same. For detailed patterns, see references/iframe-모달-패턴.md reference

Measured comparison data

Same scenario (Google → Gmail → extract 15 emails), Chrome Beta CDP 9222:

Item agent-browser playwright-cli Multiplier
eval average speed 189ms 1,249ms 6.6x
Total command time 1,521ms 6,257ms 4.1x
Total output bytes 1,918B 4,343B 2.3x
eval output format Result value only Result + code + tab list + page info -

Core rules

  1. eval first -- For data reading, clicking, and input, try eval first. Use snapshot only when you do not know the selector.
  2. Confirm the working tab with the user -- CDP cannot know the user's focused tab. tab listand explicitly confirm the tab number.
  3. For a new tab, tab new URL -- open URL --new-tab Do not use (overwrites existing tab).
  4. snapshot은 -i -c Required -- Do not take a full snapshot. Token explosion.
  5. eval only for simple expressions -- No IIFE; separate multiple actions into separate eval calls.
  6. ref changes every time -- If the DOM changes, the previous ref becomes invalid; take a snapshot again.
  7. iframe → playwright-cli fallback -- agent-browser cannot access inside iframes.
  8. connect 9222 required -- Check the CDP connection before every operation.
  9. Fixed profile path -- When launching Chrome Beta, --user-data-dirmust be $HOME/.chrome-beta-e2e-profile. Even if an error occurs, do not change the path.
  10. eval click()은 isTrusted:false -- May be ignored on secure forms. If it does not work, snapshot → click ref.
  11. For Flutter sites, calling the API directly is the strongest -- Extract the method from dart.js → call fetch. Saves 99% of tokens compared with UI navigation. (see references/Flutter-web-app-patterns.md)
  12. Detect session disconnection -- If eval fails, rerun connect 9222. Do not repeat connect every time.
  13. screenshot + Read absolutely prohibited -- Opening an image = thousands of tokens. Replace with eval "body.innerText.substring(0,200)".
  14. Before manual native dialog verification, check the resident CDP session -- agent-browser If the daemon is attached, alert/confirmmay close immediately. Rather than a new tab/refresh, lsof → ps → killtakes priority. (references/native-dialog-caution.md)
  15. Scenarios that intercepted a dialog must be reverted -- window.alert/confirm overrides, global dialog listeners, and forced dismiss code must be released in finally. (references/native-dialog-caution.md)
  16. Switching to an external service requires a fallback -- click() Use not a single line but the load + 지연 + 직접 이동 fallback pattern. (references/external-service-link-switching-patterns.md)
  17. For debugging/reproduction, E2E_TAB_ID takes priority -- --tab N indexes can fluctuate. For reproducing a tab at a specific URL, targetId fixing is safer. (references/tab-selection-patterns.md)
  18. Do not operate before checking the tab number -- tab listis a required gate. Before the user selects a tab, do not open/tab/eval/click/fill/press run.
  19. For sensitive login/profile tasks, analyze first -- For Google/OAuth/admin/payment/pages that maintain an existing session, perform analysis only and proceed with execution after user confirmation.
  20. Do not bypass the browser at the OS level -- agent-browser If open -afails, do not call another browser or run while ignoring the profile. Report and wait. (references/fixed-profile-enforced-gate-patterns.md)
  21. Authentication by the user, session use by the AI -- Login/2FA/security verification is performed directly by the user in the Chrome Beta fixed profile, and the AI takes over only the completed session to work. (references/fixed-profile-enforced-gate-patterns.md)
  22. Angular/React fill failure → add event dispatch -- If the button is [disabled] after fill, eval dispatchEvent(new Event('input', {bubbles:true}))must be attempted. If it still does not work, also dispatch a change event. (references/SPA-framework-input-patterns.md)
  23. re-verify title immediately after tab N -- tab list → user response → after tab N, must eval "document.title + ' | ' + location.href"verify that the tab is correct. If it differs from expectations, re-output tab list and reselect.
  24. Block OS file dialogs via interception -- CDP Escape key event and agent-browser press Escape are both ineffective against macOS native file dialogs (XPC services). Page.setInterceptFileChooserDialog(enabled:true) → agent-browser click → Page.fileChooserOpened → setFileInputFiles(backendNodeId) block the OS dialog itself in that order. (references/SPA-framework-input-patterns.md §3)

Reference documents

Document Purpose
references/agent-browser-명령어.md agent-browser full command reference
references/playwright-cli-명령어.md fallback only (when agent-browser is unsupported, e.g. iframes)
references/iframe-모달-패턴.md jQuery UI Dialog + iframe modal access patterns
references/Flutter-웹앱-패턴.md Flutter web app detection / accessibility / direct API calls
references/토큰-최적화-실측데이터.md Measured token cost comparison table per command + budget guide
references/native-dialog-주의사항.md Native dialog auto-dismiss issue in CDP persistent sessions + inspection procedure
references/외부서비스-링크전환-패턴.md Fallback patterns when switching to external service links
references/탭-선택-패턴.md --tab N vs E2E_TAB_ID selection criteria
references/SPA-프레임워크-입력패턴.md Angular/React controlled input event dispatch, hidden file input TreeWalker traversal, CDP Input.insertText
references/고정프로필-강제게이트-패턴.md Stop conditions and no-bypass rules for sensitive session/fixed profile work