Use the Chrome DevTools Protocol for web page data retrieval and analysis.
Workflow (AI executes automatically)
When the user uses this skill, the AI shouldautomatically complete all of the following steps, rather than having the user perform them manually:
1. Automatically start Chrome remote debugging
The AI automatically detects the operating system and starts Chrome:
Windows:
start chrome --remote-debugging-port=9222 --user-data-dir="%TEMP%\chrome-devtools-profile" [URL]
Important note: Use a fixed
--user-data-dir(such aschrome-devtools-profile), do not use a different folder each time. This way Chrome will remember the login state, cookies, and cache, and will start faster next time.
macOS:
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222 --user-data-dir="$TMPDIR/chrome-profile-stable" [URL]
Linux:
google-chrome --remote-debugging-port=9222 --user-data-dir="/tmp/chrome-profile-stable" [URL]
2. Automatically start the MCP server
The AI automatically starts the MCP server (running in the background):
npx -y chrome-devtools-mcp@latest --browser-url=http://127.0.0.1:9222
3. Execute the operation requested by the user
Use the following MCP tools to complete the user's task:
mcp__chrome-devtools__navigate_page- Navigate to a pagemcp__chrome-devtools__wait_for- Wait for the page to loadmcp__chrome-devtools__take_snapshot- Get a page snapshotmcp__chrome-devtools__evaluate_script- Execute JavaScript to extract datamcp__chrome-devtools__screenshot- Take a screenshotmcp__chrome-devtools__list_network_requests- List network requests
Standard operating procedure (used by the AI)
When the user asks to open/analyze/scrape a web page, the AI executes in the following order:
1. 启动 Chrome(调试模式)
2. 启动 MCP 服务器
3. 导航到目标 URL
4. 等待页面加载
5. 执行用户请求的操作(截图/提取数据/监控网络等)
6. 返回结果给用户
Data extraction techniques
Extract list data:
() => {
const items = document.querySelectorAll(".item");
return Array.from(items).map((item) => ({
title: item.querySelector(".title")?.innerText,
link: item.querySelector("a")?.href,
}));
};
Extract table data:
() => {
const rows = document.querySelectorAll("table tr");
return Array.from(rows).map((row) =>
Array.from(row.querySelectorAll("td, th")).map((cell) => cell.innerText)
);
};
Common use cases
- Open a web page and take a screenshot: Start Chrome → navigate → screenshot → return result
- Product information scraping: Navigate → scroll to load → extract list
- Form automation: Navigate → fill_form → click → wait_for
- API monitoring: Navigate → list_network_requests → get_network_request
- Performance analysis: performance_start_trace → operate → performance_stop_trace
Best practices
- Automatically handle all prerequisite steps - Do not ask the user to manually start Chrome or the MCP server
- Use a fixed user data directory - Keep
--user-data-dirconsistent to avoid reloading every time - First use
take_snapshotto understand the page structure - For complex extraction, use
evaluate_scriptto execute JavaScript - Ensure the returned data is JSON-serializable
- When handling dynamic content, use
wait_for
Technical principles
Chrome DevTools Protocol (CDP) communication methods
| Method | Protocol | Purpose |
|---|---|---|
| HTTP | http://127.0.0.1:9222/json/list |
Get the page list and basic information |
| WebSocket | ws://127.0.0.1:9222/devtools/page/{pageId} |
Control pages in real time and execute scripts |
WebSocket communication flow
import asyncio
import websockets
import json
async def cdp_example():
# 1. 连接到 Chrome DevTools WebSocket
ws_url = "ws://127.0.0.1:9222/devtools/page/{pageId}"
async with websockets.connect(ws_url) as ws:
# 2. 启用 Runtime 域
await ws.send(json.dumps({
"id": 1,
"method": "Runtime.enable"
}))
# 3. 执行 JavaScript
await ws.send(json.dumps({
"id": 2,
"method": "Runtime.evaluate",
"params": {
"expression": "document.title",
"returnByValue": True
}
}))
# 4. 循环接收响应,匹配 id
while True:
msg = await ws.recv()
data = json.loads(msg)
if data.get('id') == 2: # 匹配请求 id
return data['result']['result']['value']
Key considerations
- Response matching: WebSocket receives multiple kinds of messages (
consoleAPICalled、executionContextCreated, etc.), and requests and responses must be matched viaidMatch requests and responses - Wait for the page to load: After performing an operation, use
await asyncio.sleep(2)or wait for a specific element - Encoding issues: Windows terminals require
sys.stdout.reconfigure(encoding='utf-8')to correctly display Chinese
Practical Experience and Pitfall Records
Issue 1: Windows Chinese Output Garbled Text
Symptom: Scraped Chinese content displays as garbled text or throws UnicodeEncodeError: 'gbk' codec can't encode character
Cause: The Windows terminal uses GBK encoding by default, which conflicts when Python outputs UTF-8 Chinese
Solution: Add encoding configuration at the beginning of the Python script:
import sys
sys.stdout.reconfigure(encoding='utf-8')
Important: All example scripts (baidu_search_example.py、cdp_helper.py) have this encoding configuration added, and running them directly will not produce garbled text.
Issue 2: HTML Character Entity Escaping
Symptom: In JavaScript code, => arrow functions are escaped as =>
Cause: The JSON standard performs Unicode escaping on certain characters
Solution: This is normal and does not affect actual execution. If you need to handle it in Python, you can use:
import codecs
decoded = codecs.decode(string, 'unicode_escape')
Issue 3: Obtaining the Page ID
Symptom: You do not know the WebSocket URL of the current page Solution: Obtain the page list through the HTTP interface:
curl -s http://127.0.0.1:9222/json/list
Then extract the webSocketDebuggerUrl field
Note: The page ID is dynamically generated and changes every time Chrome starts. It should be obtained dynamically in the code rather than hardcoded:
import urllib.request
import json
def get_ws_url():
req = urllib.request.Request('http://127.0.0.1:9222/json/list')
with urllib.request.urlopen(req) as response:
pages = json.loads(response.read().decode())
return pages[0]['webSocketDebuggerUrl'] if pages else None
Issue 4: Cross-Platform Compatibility Issues with Windows Wait Commands
Symptom: Using timeout /t 3 /nobreak in Git Bash reports an error invalid time interval
Cause: timeout is a Windows CMD command, and its syntax is incompatible in Bash
Solution: Use a cross-platform waiting solution:
# Windows CMD
timeout /t 3 /nobreak >nul
# Git Bash / Linux
sleep 3
# 跨平台通用(Windows)
ping -n 4 127.0.0.1 > nul
Issue 5: Claude Code File Writing Restrictions
Symptom: When directly using the Write tool to create a new file, it prompts "File has not been read yet" Cause: Claude Code requires that a file must be read before it can be written (security mechanism) Solution:
- First use the Read tool to read the file (if it does not exist, an error will be reported, and you need to use Bash to create an empty file)
- Or use Bash's
echoorcatcommand to write directly - Or use a Python script to dynamically generate and execute code
Issue 6: Incomplete Extraction of Dynamic Content
Symptom: Some fields in the extracted search results (such as summaries) are empty Cause:
- The page uses dynamic loading and requires scrolling to display all content
- Different websites use different class names
- Some content is loaded asynchronously
Solution:
- Use multiple selectors for matching:
const abstract = item.querySelector('.c-abstract, .abstract, [class*="abstract"]')?.innerText;
- Scroll the page to load more content:
window.scrollTo(0, document.body.scrollHeight);
- Increase the waiting time to ensure asynchronous content finishes loading
Issue 7: Foreground vs Background Browser Operation
CDP supports two operation modes, choose according to user needs:
Background operation (default):
- Send JavaScript commands through WebSocket to directly manipulate the DOM
- The browser interface will not display the operation process (users cannot see actions such as input and clicks)
- Suitable for scenarios such as data scraping and automated testing that do not require visual feedback
- Code example:
# 直接设置输入框的值(用户看不到输入过程)
await ws.send(json.dumps({
"method": "Runtime.evaluate",
"params": {
"expression": "document.querySelector('#kw').value = '美女';"
}
}))
Foreground operations (visual):
- Use the CDP Input domain to simulate real user input
- The browser will display the complete operation process (keyboard input, mouse clicks, etc.)
- Suitable for demos, teaching, and scenarios where the user needs to observe the operation process
- Code example:
# 1. 先聚焦输入框
await ws.send(json.dumps({
"method": "Runtime.evaluate",
"params": {
"expression": "document.querySelector('#kw').focus();"
}
}))
# 2. 模拟键盘输入(用户能看到逐字输入)
for char in "美女":
await ws.send(json.dumps({
"method": "Input.dispatchKeyEvent",
"params": {
"type": "char",
"text": char
}
}))
await asyncio.sleep(0.1) # 模拟真实打字间隔
# 3. 模拟点击搜索按钮(用户能看到点击效果)
await ws.send(json.dumps({
"method": "Input.dispatchMouseEvent",
"params": {
"type": "mousePressed",
"x": 100,
"y": 200,
"button": "left"
}
}))
Selection recommendations:
| Scenario | Recommended mode | Reason |
|---|---|---|
| Data scraping | Background | Fast, no visualization needed |
| Automated testing | Background | Stable and reliable, unaffected by UI |
| Search-type tasks | Background | Directly constructing the URL is simpler and more reliable |
| Operation demo | Foreground | The user can see the complete process |
| Teaching demo | Foreground | Easy to observe and understand |
| Form filling | Either is fine | Background is faster, foreground is more intuitive |
Recommended solution for search-type tasks:
- Background method(recommended): Directly construct the search URL, such as
https://www.baidu.com/s?wd=关键词 - Foreground method: Use only when the user explicitly requests "visualization" or "being able to see the operation process"
Question 8: Best practices for search-type tasks
Scenario: The user requests "search for a certain keyword on Baidu/Google and scrape the results"
Solution comparison:
| Solution | Implementation method | Advantages | Disadvantages | Recommendation level |
|---|---|---|---|---|
| A | Directly access the search URL in the background | Simple, fast, stable | The user cannot see the process | ⭐⭐⭐⭐⭐ |
| B | Simulate input and clicks in the foreground | Visual, with feedback | Complex, slow, requires handling coordinates | ⭐⭐⭐ |
Recommended solution A (background method):
# 直接构造搜索URL,一步到位
url = f"https://www.baidu.com/s?wd={urllib.parse.quote(keyword)}"
await client.navigate(url)
await asyncio.sleep(3) # 等待加载
results = await client.evaluate(extract_script)
When to use solution B (foreground method):
- The user explicitly requests "visual operation" or "let me see the process"
- Teaching demo scenario
- Need to demonstrate the automated operation process
Precautions for the foreground method:
- Must first
focus()input box - A typing interval of 0.1-0.15 seconds is most suitable
- The coordinates of the Baidu search button are approximately x:650, y:230 (1920x1080 resolution)
- Need to enable the Input domain:
Input.enable
Detailed steps for foreground text input(using the Baidu search box as an example):
# 步骤 1: 启用 Input 域(必须先启用才能发送键盘事件)
await ws.send(json.dumps({
"id": 1,
"method": "Input.enable"
}))
# 步骤 2: 聚焦输入框(关键步骤,否则键盘事件可能无效)
await ws.send(json.dumps({
"id": 2,
"method": "Runtime.evaluate",
"params": {
"expression": "document.querySelector('#kw').focus();", # '#kw' 是百度搜索框ID
"returnByValue": True
}
}))
await asyncio.sleep(0.5) # 等待聚焦完成
# 步骤 3: 清空输入框(可选,确保输入框为空)
await ws.send(json.dumps({
"id": 3,
"method": "Runtime.evaluate",
"params": {
"expression": "document.querySelector('#kw').value = '';",
"returnByValue": True
}
}))
# 步骤 4: 逐字输入文字(用户能看到逐字输入动画)
text = "hello 你好呀"
for i, char in enumerate(text):
await ws.send(json.dumps({
"id": 10 + i,
"method": "Input.dispatchKeyEvent",
"params": {
"type": "char", # 输入字符
"text": char # 要输入的字符
}
}))
await asyncio.sleep(0.15) # 打字间隔,让用户能看到动画
Key points:
type: "char"Used to input ordinary characters- Must first
focus()input box, otherwise the input may be invalid - The interval time is recommended to be 0.1-0.2 seconds; too short may miss characters, too long makes users wait a long time
- Supports Chinese input, no additional encoding processing required
Question 9: Form automation execution order
Correct order:
- Enable Runtime → 2. Fill input boxes → 3. Click buttons → 4. Wait for the page to load → 5. Extract data
Example Code(Baidu search backend method):
# 1. 填充搜索词
await ws.send(json.dumps({
"id": 2,
"method": "Runtime.evaluate",
"params": {
"expression": "document.querySelector('#kw').value = '搜索词';",
"returnByValue": True
}
}))
# 2. 点击搜索按钮
await ws.send(json.dumps({
"id": 3,
"method": "Runtime.evaluate",
"params": {
"expression": "document.querySelector('#su').click();",
"returnByValue": True
}
}))
# 3. 等待加载
await asyncio.sleep(3)
# 4. 提取结果
await ws.send(json.dumps({
"id": 4,
"method": "Runtime.evaluate",
"params": {
"expression": """
(() => {
const items = document.querySelectorAll('.result');
return Array.from(items).map(item => ({
title: item.querySelector('h3')?.innerText,
link: item.querySelector('a')?.href
}));
})()
""",
"returnByValue": True
}
}))
User Guide
How to Use This Skill
Users can trigger this skill in the following ways:
- State the requirement directly - Describe what you want to do with the webpage
- Use
/chrome-devtools-skillcommand - Explicitly invoke the skill - Mention webpage operations - Any keywords involving "open webpage, scrape data, screenshot", etc.
Prompt Examples
The following are some prompt templates users can use:
Basic Operations
打开 https://www.example.com 并截图
抓取 https://www.example.com 的页面内容
分析一下这个网页:https://github.com/xxx/xxx
Data Extraction
打开 https://www.jd.com,提取首页所有商品标题和价格
访问 https://www.zhihu.com/explore,获取热门问题列表
抓取 https://www.example.com 页面中的所有链接
Monitoring and Analysis
打开 https://www.example.com,监控页面加载的所有 API 请求
分析 https://www.example.com 的页面性能
访问 https://www.example.com,提取页面中的所有图片地址
Automated Operations
打开 https://www.example.com,在搜索框输入"关键词"并搜索
访问 https://www.example.com,点击"加载更多"按钮,抓取所有列表数据
Prompt Techniques
- Specify URL - Provide the complete webpage address
- State the goal - Clearly describe what data you want to obtain or what operation you want to perform
- Specify the format - If a specific format is needed, you can specify it (e.g., "return in table form")
- Break complex tasks into steps - For multi-step tasks, you can describe them step by step
Example Dialogue
User: Help me open Baidu and see what's on the trending list
AI: [Automatically launch Chrome → Open Baidu → Extract trending data → Return results]
User: Scrape the top 10 news titles and links from https://news.ycombinator.com the top 10 news headlines and links
AI: [Automatically launch Chrome → Open Hacker News → Extract news data → Return table]
User: Analyze which APIs are loaded on the Taobao homepage
AI: [Automatically launch Chrome → Open Taobao → Monitor network requests → Return API list]
Reference Resources
- Example Scripts: See
scripts/directorybaidu_search_example.py- Complete Baidu search examplecdp_helper.py- CDP client wrapper classREADME.md- Quick start guide
Chrome DevTools MCP Configuration Information
If the user askschrome-devtools MCPhow to configure, provide the following JSON configuration information:
{
"mcpServers": {
"chrome-devtools": {
"command": "npx",
"args": [
"chrome-devtools-mcp@latest",
"--browser-url=http://127.0.0.1:9222"
]
}
}
}
// Test modification // Another test modification // Test modification for pre-push approach