Call the Python Playwright script (scripts/cnki_search.py) to execute the full automated search workflow, without needing MCP tools to operate the browser.
Step 0: Parse user input and formulate a search strategy (reverse onion-peeling method)
0.1 Identify input type
Type 1: Single keyword(the user gives one word or concept)
→ Connect that word and its synonyms and coordinate terms with + and fill them into a single search box.
Example: the user says "emotional labor" → 情感劳动 + 情绪劳动 + 情感工作
Type 2: Research topic / research question(the user gives a complete statement containing multiple concepts)
→ Extract from the topic 2-4 core concepts, expand synonyms/coordinate terms for each concept separately, and treat each as a search group,with the relationship between groups being AND(combined search).
Example: the user says "the suspension and reconstruction of digital governance in old communities of megacities" → extract three groups of core concepts:
- Group 1 (setting):
老旧社区 + 老旧小区 + 城市社区 - Group 2 (issue):
数字治理 + 数字化治理 + 数字技术治理 - Group 3 (mechanism):
技术悬浮 + 数字悬浮 + 技术失灵
The three groups form an AND combined search, focusing on the intersecting literature of "old communities × digital governance × suspension mechanism".
Note: Do not search several concepts of a topic independently and then merge them, as that will retrieve a large amount of irrelevant literature. Only an AND combined search can focus on the core literature.
0.2 Pre-analysis of broader concepts (key step)
Before filling in terms, complete the following judgment for each concept group:
Determine whether the group of terms is "too precise/niche"：
- Emerging theoretical terms (such as "technological suspension" and "institutional suspension")
- Extremely specific descriptions of a setting or phenomenon (such as "old communities" and "urban villages")
- Policy/practice concepts that have appeared only in recent years (such as "digital village" and "reverse parenting")
If a group of terms falls into the above categories,immediately add broader concepts within that group(using + OR relationship); do not wait until the search fails before expanding.
| Concept type | Precise term | Broader concept/fallback term to proactively include |
|---|---|---|
| Emerging theoretical term | Technological suspension + digital suspension | Technological failure + digital divide + grassroots digitalization |
| Specific setting | Old communities + old residential areas | Urban communities + community governance |
| New policy concept | Digital village | Rural digitalization + rural revitalization + rural informatization |
| Specific mechanism | Institutional suspension | Institutional failure + institutional implementation + grassroots governance |
Rule: Better to make the initial phrase set slightly broader than to wait until retrieval returns 0 before thinking of expanding. Broader concepts should be included in the same group's
+OR list at the design stage.
0.3 Reverse Onion-Peeling Strategy (automatic downgrade when results are insufficient)
First useAND retrieval of all concept groups(the most focused level). If the number of retrieval results is too small (< 20 records), automatically reduce the number of AND groups,keep at least 2 AND groups, and do not automatically drop to 1 group:
| Level | Strategy | Applicable scenario |
|---|---|---|
| Level 1 (most focused) | AND of all concept groups | Intersectional research, emerging topics |
| Level 2 | Remove the least important group, keep the core N-1 groups with AND | Level 1 results < 20 records |
| Level 3 (bottom line) | Keep the 2 most core groups with AND | Level 2 results still < 20 records |
Principle for the order of removing groups: first remove the concept groups least related to the research question (such as modifying concepts or contextual concepts), and keep the most core theoretical/mechanism concept groups until the end.
0.4 Confirm with the user
List the proposed retrieval groupings and the current level strategy, and execute only after the user confirms. Format example:
检索策略：第1层（3组 AND 联合检索）
- 组1（场所）：老旧社区 + 老旧小区 + 城市社区 + 社区治理
- 组2（议题）：数字治理 + 数字化治理 + 数字技术治理
- 组3（机制）：技术悬浮 + 数字悬浮 + 技术失灵 + 基层数字化
若结果 < 20篇，将自动降至第2层（去掉组3，保留组1 AND 组2）
若仍 < 20篇，保留2组AND并停止降级，提示调整关键词
0.5 Trigger conditions for result reflection
When the script reports the following situations,you must stop and rethink the keywords, and must not export directly：
| Situation | Diagnostic direction |
|---|---|
| 2 groups with AND return 0 results | Both groups of terms are too specific → replace each with higher-level broader concepts |
| 2 groups with AND return < 5 records | A group of terms is too obscure → identify which group is narrower and replace that group first |
| 2 groups with AND return 5-19 records | Boundary case → try adding 1-2 broader concepts to one group and search again |
Bottom-line principle: Better to modify the keywords and rerun than to accept and directly export a "2 groups with AND < 20 records" result. The result range of 1 group with AND is too broad and has no literature-focusing value.
Step 0.5: Check Chrome debugging mode
Before execution, check whether Chrome is already running in debugging mode (the script will automatically try to connect):
curl -s http://localhost:9222/json/version 2>&1
- If it returns JSON(including
"Browser": "Chrome/..."): Chrome is ready; proceed directly to Step 1. - If it returns an error/empty: execute the following command to start Chrome, wait 2 seconds, and then check again:
open -a "Google Chrome" --args --remote-debugging-port=9222
sleep 2
curl -s http://localhost:9222/json/version 2>&1
If there is still no response, prompt the user to manually start Chrome in debugging mode and then continue.
Note: connecting to an existing Chrome can preserve the logged-in CNKI session and skip the login step.
Step 1: Call the Python script to execute the retrieval (reverse onion-peeling execution)
Start retrieval from Level 1, and decide whether to downgrade based on the number of results.
Level 1 (all groups with AND)：
python3 /Users/songyiping/.claude/skills/cnki-advanced-search/scripts/cnki_search.py \
--keywords "老旧社区 + 老旧小区 + 城市社区" \
--keywords "数字治理 + 数字化治理 + 数字技术治理" \
--keywords "技术悬浮 + 数字悬浮 + 技术失灵" \
--max-results 100 \
--port 9222
If results < 20 records → downgrade to Level 2 (remove the least important group)：
python3 /Users/songyiping/.claude/skills/cnki-advanced-search/scripts/cnki_search.py \
--keywords "老旧社区 + 老旧小区 + 城市社区" \
--keywords "数字治理 + 数字化治理 + 数字技术治理" \
--max-results 100 \
--port 9222
If still < 20 records → downgrade to Level 3 (single group)：
python3 /Users/songyiping/.claude/skills/cnki-advanced-search/scripts/cnki_search.py \
--keywords "数字治理 + 数字化治理 + 社区数字化" \
--max-results 100 \
--port 9222
Before each downgrade, explain to the user: "Level N results are X records, fewer than 20, automatically downgrading to Level N+1 retrieval."
The script will automatically handle：
- Connect to the existing Chrome (CDP port 9222) or launch a new Chromium
- Navigate to the CNKI advanced search page
- CAPTCHA detection (pause when it appears, wait for the user to press Enter in the terminal to confirm completion)
- Login check (pause and wait for the user to log in when not logged in)
- Select academic journals, check CSSCI, enter keywords
- Run the search, sort by citation count in descending order, 50 per page
- Select all and export page by page, up to 100 articles
- Parse the data and save as Excel
Script output example：
===== 知网高级检索自动化工具 =====
关键词分组: ['老旧社区 + 老旧小区', '数字治理 + 数字化治理', '技术悬浮 + 数字悬浮']
最大结果数: 100
...
[✓] 检索完成！共提取 87 篇论文
[✓] 文件已保存：/Users/songyiping/Downloads/知网检索_老旧社区_20260406_1523.xlsx
Step 2: Independent supplementary search (indirectly related literature) — required step
After the main combined search is completed,mustperform an independent supplementary search, with the purpose of obtaining indirectly related literature that is "not directly at the intersection, but has theoretical support and background value for the topic."
2.1 Design principles for supplementary search
From the concept groups already extracted in Step 0, separately break out 2-3 supplementary searches：
| Supplementary search type | Source concept | Purpose |
|---|---|---|
| Research object background literature | Search the "research object group" separately (relax qualifiers) | Obtain the historical context, policy background, and macro descriptive literature of this group |
| Core theoretical literature | Search the "theory/mechanism concept group" separately | Obtain theoretical foundation literature supporting the topic, without restricting the research object |
| Extended topic literature(optional) | Take the secondary concepts or higher-level themes in the topic | Cross-boundary literature covering the boundaries of the topic |
Important: each supplementary search uses 2 groups of AND(do not do a single broad group search), and a certain degree of focus must still be maintained. The purpose of supplementary search is to "switch to another facet," not to "expand infinitely."
2.2 Show the supplementary search plan to the user
In the strategy confirmation of Step 0.4, list the supplementary search plan at the same time, for example:
【主体联合检索】第1层（2组 AND）
- 组1（对象）：网约车司机 + 快车司机 + 平台司机
- 组2（议题）：平台劳动 + 零工经济 + 意义建构 + 身份认同
【独立补充检索】（主体检索完成后必做）
- 补充检索A（研究对象背景）：出租车 + 网约车 × 职业认同 + 身份认同 + 劳动体验
- 补充检索B（核心理论）：平台劳动 + 零工经济 + 算法管理 × 主体性 + 劳动控制 + 抵抗
2.3 Execute supplementary search
Each supplementary search likewise uses cnki_search.py, and the results are saved to a separate Excel file:
# 补充检索A：研究对象背景文献
python3 /Users/songyiping/.claude/skills/cnki-advanced-search/scripts/cnki_search.py \
--keywords "出租车 + 网约车 + 出租车司机 + 网约车司机" \
--keywords "职业认同 + 身份认同 + 劳动体验 + 生计" \
--max-results 60 \
--port 9222
# 补充检索B：核心理论文献
python3 /Users/songyiping/.claude/skills/cnki-advanced-search/scripts/cnki_search.py \
--keywords "平台劳动 + 零工经济 + 平台工人 + 数字劳动 + 算法管理" \
--keywords "主体性 + 劳动控制 + 抵抗 + 意义建构 + 身份认同" \
--max-results 80 \
--port 9222
When the supplementary search results are fewer than 20 articlesdo not force downgrade(because indirectly related literature is expected to be small in number), but if the result is 0, the higher-level concept needs to be replaced and retried.
Step 3: Aggregate and merge — output a unified classified Excel
After all searches (main + supplementary) are completed,mustuse a Python script to merge and deduplicate all results, generating a unified classified Excel.
3.1 Merge script template
import openpyxl
from openpyxl.styles import Font, PatternFill, Alignment
from collections import Counter
# 1. 定义各批次文件及其类别标签
files = {
"直接相关（[主体对象]×[核心议题]）": "/Users/songyiping/Downloads/知网检索_XXX.xlsx",
"间接相关-[背景类型]（[对象]×[议题]）": "/Users/songyiping/Downloads/知网检索_YYY.xlsx",
"间接相关-[理论类型]（[理论]×[机制]）": "/Users/songyiping/Downloads/知网检索_ZZZ.xlsx",
}
# 2. 读取、去重（按"标题"列）
all_rows = []
seen_titles = set()
title_idx = 1 # Excel列: 序号(0), 标题(1), 作者(2), 来源期刊(3), 发表时间(4), 摘要(5)
for category, filepath in files.items():
wb = openpyxl.load_workbook(filepath)
ws = wb.active
rows = list(ws.iter_rows(values_only=True))
for row in rows[1:]:
title = str(row[title_idx]).strip() if row[title_idx] else ""
if title and title not in seen_titles:
seen_titles.add(title)
all_rows.append((category,) + tuple(row[1:])) # 去掉原序号，前置类别
# 3. 写入汇总 Excel
wb_out = openpyxl.Workbook()
ws_out = wb_out.active
ws_out.title = "文献汇总"
headers = ["序号", "文献类别", "标题", "作者", "来源期刊", "发表时间", "摘要"]
ws_out.append(headers)
# 表头样式
for cell in ws_out[1]:
cell.font = Font(bold=True, color="FFFFFF", size=11)
cell.fill = PatternFill(fill_type="solid", fgColor="2F5496")
cell.alignment = Alignment(horizontal="center", vertical="center", wrap_text=True)
# 各类别颜色（按类别数量自动分配，以下为示例3类）
category_colors = {
list(files.keys())[0]: "D9E1F2", # 蓝：直接相关
list(files.keys())[1]: "E2EFDA", # 绿：间接相关A
list(files.keys())[2]: "FFF2CC", # 黄：间接相关B
}
for seq, row_data in enumerate(all_rows, 1):
category = row_data[0]
ws_out.append((seq,) + row_data)
fill_color = category_colors.get(category, "FFFFFF")
for cell in ws_out[ws_out.max_row]:
cell.fill = PatternFill(fill_type="solid", fgColor=fill_color)
cell.alignment = Alignment(vertical="top", wrap_text=True)
# 列宽
for i, width in enumerate([4, 30, 20, 16, 12, 60], 1):
ws_out.column_dimensions[openpyxl.utils.get_column_letter(i)].width = width
ws_out.column_dimensions["B"].width = 30 # 类别列
# 行高
for row in ws_out.iter_rows(min_row=2):
ws_out.row_dimensions[row[0].row].height = 60
ws_out.freeze_panes = "A2"
# 分类统计 sheet
ws_summary = wb_out.create_sheet("分类统计")
ws_summary.append(["文献类别", "篇数"])
for cat, cnt in Counter(r[0] for r in all_rows).items():
ws_summary.append([cat, cnt])
ws_summary.append(["合计", len(all_rows)])
for cell in ws_summary[1]:
cell.font = Font(bold=True)
output_path = f"/Users/songyiping/Downloads/知网文献汇总_{研究主题}_{日期}.xlsx"
wb_out.save(output_path)
print(f"[✓] 已保存: {output_path} （共{len(all_rows)}篇，去重后）")
3.2 Color conventions
| Category | Color | Meaning |
|---|---|---|
| Directly related | Blue D9E1F2 |
Hits from the main combined search, focused on the core of the topic |
| Indirectly related-background | Green E2EFDA |
Historical background literature on the research object/context |
| Indirectly related-theory | Yellow FFF2CC |
Foundational supporting literature for the theory/mechanism |
| Indirectly related-extension | Orange FCE4D6 |
Other extended-perspective literature (if there is a 4th category) |
Step 4: Report results
Output a retrieval completion report, informing the user of:
- The retrieval level and number of articles for each round
- Total number of articles after deduplication
- Distribution by category (direct/indirect)
- Summary file path
Open files:
open "/Users/songyiping/Downloads/知网文献汇总_*.xlsx"
For more AI agent learning materials and tips, welcome to follow:
Troubleshooting common issues
| Issue | Solution |
|---|---|
CDP连接失败 |
The script will automatically fall back to launching a new browser, but manual login to CNKI is required |
检测到验证码 |
The script pauses and prints a prompt; after completing verification in the browser, press Enter in the terminal |
知网导出功能需要登录 |
The script pauses; after logging in through the browser, press Enter |
解析失败 |
The original text is saved to ~/Downloads/cnki_raw_export_*.txt, which can be checked manually |
检索结果为0 |
Adjust the keywords, reduce synonym combinations, or broaden the search scope |
| Need to debug delays | Add --delay 1500(increase the wait time between operations, in milliseconds) |
Dependencies
playwright(installed)openpyxl(installed)- If the script prompts that a browser is missing: run
playwright install chromium