From zero to release: API discovery → auth strategy → write adapter → test and verify.
Choose a path first
| Situation | Go here |
|---|---|
| You only need to generate one command for one specific page | opencli-oneshot skill |
| You want to let the machine try automatically first | opencli generate <url> [--goal <goal>], and come back if it fails |
| New site / multiple commands / oneshot stuck | Keep reading this document |
| The output is going to be submitted as a PR | This document + clis/<site>/ + npm run build |
| Just for local private use, no PR | This document + ~/.opencli/clis/<site>/ |
Core workflow
┌─────────────┐ ┌─────────────┐ ┌──────────────┐ ┌────────┐
│ 1. 发现 API │ ──▶ │ 2. 选择策略 │ ──▶ │ 3. 写适配器 │ ──▶ │ 4. 测试 │
└─────────────┘ └─────────────┘ └──────────────┘ └────────┘
browser explore cascade TS cli() API verify
Required reading for AI Agents: you must explore with a browser
[!CAUTION] You must open the target website in a browser to explore! Don't rely on static analysis alone. Many APIs arelazy-loaded—deep data such as subtitles, comments, and follow lists is only triggered after clicking.
Browser exploration workflow
| Step | Command | What it does |
|---|---|---|
| 0. Open the page | opencli browser open <url> |
Navigate to the target page and start capturing |
| 1. Observe elements | opencli browser state |
View interactive elements (buttons/tabs), with [N] indexes |
| 2. First capture | opencli browser network |
List the captured JSON API requests |
| 3. Simulate interaction | opencli browser click <N> |
Click buttons to trigger lazy-loaded APIs |
| 4. Second capture | opencli browser network |
Find the newly triggered APIs |
| 5. View the response | opencli browser network --detail <N> |
View the full response body |
| 6. Verify the API | opencli browser eval "fetch(...).then(r=>r.json())" |
Confirm the API is reproducible |
Common mistakes
| ❌ Wrong approach | ✅ Correct approach |
|---|---|
Only use opencli explore, and wait for the results |
Use opencli browser open to browse proactively |
| Write code directly without looking at browser requests | First opencli browser network to confirm the API |
| Capture directly after opening the page | Use opencli browser click to simulate interaction and trigger lazy loading |
| Give up when HTTP 200 but the data is empty | Check whether signature or Cookie authentication is required (fake 200 risk control) |
Rely on __INITIAL_STATE__ to get all the data |
__INITIAL_STATE__ only has first-screen data; deep data requires calling the API |
opencli browser network is empty |
Re- open refresh and capture; or check whether the API is on a separate domain |
Hands-on example: implement a "follow list" adapter in 5 minutes
opencli browser open https://space.bilibili.com/{uid}/fans/follow
opencli browser network
# [0] GET 200 /x/relation/followings?vmid={uid}&pn=1&ps=24
opencli browser network --detail 0
# 确认数据结构:{ code: 0, data: { total: 1342, list: [{mid, uname, ...}] } }
opencli browser eval "fetch('/x/relation/followings?vmid=137702077&pn=1&ps=5', {credentials:'include'}).then(r=>r.json())"
# → 有数据,结论:Tier 2 Cookie,写 following.js
Step 1: Discover the API
Main path: proactive browser exploration
Use the workflow above to open the page → simulate interaction → capture. Pay attention to:
- URL pattern:
/api/v2/hot?limit=20→ the endpoint to call - Method:GET / POST
- Request Headers: Cookie? Bearer? Custom signature headers (X-s, X-t)?
- Response Body: data path (
data.items、data.list)
Advanced shortcuts (try in order of priority)
- Suffix brute-force method (
.json): Reddit, Xueqiu, etc., add.jsonto the URL to get REST data directly (Tier 2 instant win) - Global state method (
__INITIAL_STATE__): SSR sites (Bilibili, Xiaohongshu) have homepage data mounted on window - Active interaction trigger method: Lazy-loaded APIs require clicking buttons ("CC", "Expand All") to trigger
- Framework Store truncation: Vue + Pinia sites, Store Actions complete signing for you
- XHR/Fetch interception: Last resort, use
installInterceptorpacket capture
Framework detection
opencli browser eval "(()=>{
const vue3 = !!document.querySelector('#app')?.__vue_app__;
const pinia = vue3 && !!document.querySelector('#app').__vue_app__.config.globalProperties.\$pinia;
const react = !!window.__REACT_DEVTOOLS_GLOBAL_HOOK__;
return JSON.stringify({vue3, pinia, react});
})()"
Vue + Pinia → Can use Store Actions to bypass signing (Tier 4).
Step 2: Choose authentication strategy
opencli cascade https://api.example.com/hot # 自动探测
Strategy decision tree
fetch(url) 直接能拿到?
→ ✅ Tier 1: public(browser: false,~1s)
→ ❌ fetch(url, {credentials:'include'}) 带 Cookie 能拿到?
→ ✅ Tier 2: cookie(最常见)
→ ❌ localStorage 有 token,Bearer header 能拿到?
→ ✅ Tier 2.5: localStorage Bearer(现代 SaaS 主流)
带了 Bearer 但 400 "Missing X-Xxx header"?
→ 先调 /servers 或 /workspaces 拿业务上下文 ID
→ ❌ 加 CSRF header 后能拿到?
→ ✅ Tier 3: header(如 Twitter ct0 + Bearer)
→ ❌ 网站有 Pinia/Vuex Store?
→ ✅ Tier 4: intercept(Store Action + XHR 拦截)
→ ❌ Tier 5: ui(UI 自动化,最后手段)
Strategy comparison
| Tier | Strategy | Speed | Applicable scenarios | Examples |
|---|---|---|---|---|
| 1 | public |
⚡ ~1s | Public API, no login required | Hacker News, V2EX |
| 2 | cookie |
🔄 ~7s | Cookie authentication is sufficient | Bilibili, Zhihu, Reddit |
| 2.5 | localStorage Bearer |
🔄 ~7s | JWT stored in localStorage, API on separate domain | Slock, Linear, Notion |
| 3 | header |
🔄 ~7s | Requires CSRF token or Bearer | Twitter GraphQL |
| 4 | intercept |
🔄 ~10s | Requests have complex signing | Xiaohongshu (Pinia + XHR) |
| 5 | ui |
🐌 ~15s+ | No API, pure DOM parsing | Legacy websites |
Step 2.5: Preparation
Reuse existing adapters first, don't start from scratch:
ls clis/<site>/ # 看同站点已有什么
cat clis/<site>/feed.js # 读最相似的那个
Only 3 changes needed:name, API URL, field mapping.
Bilibili platform SDK(clis/bilibili/utils.js):
fetchJson(page, url)→ Regular Cookie-tier APIapiGet(page, path, {signed, params})→ Interface with URL containing/wbi/(auto-signed)getSelfUid(page)/resolveUid(page, input)→ User UID handling
Step 3: Write adapter
All adapters uniformly use the cli() API, place into clis/<site>/<name>.js to auto-register.
Complete templates (Tier 1~4), pagination patterns, error handling conventions (throw CliError + YAML envelope)→ adapter-templates.md
Minimal structure(Tier 2 Cookie):
import { cli, Strategy } from '@jackwener/opencli/registry';
cli({
site: 'mysite',
name: 'mycommand',
description: '一句话描述',
domain: 'www.example.com',
strategy: Strategy.COOKIE,
browser: true,
args: [{ name: 'limit', type: 'int', default: 20 }],
columns: ['rank', 'title', 'value'],
func: async (page, kwargs) => {
await page.goto('https://www.example.com');
const data = await page.evaluate(`(async () => {
const res = await fetch('/api/items', { credentials: 'include' });
const d = await res.json();
return d.data?.items || [];
})()`);
return (data as any[]).slice(0, kwargs.limit).map((item, i) => ({
rank: i + 1,
title: item.title || '',
value: item.value || '',
}));
},
});
Cascading requests, tap debugging, anti-change patterns → advanced-patterns.md
Step 4: Test
Build passing ≠ Functioning correctly. Must actually run and confirm output.
Two development scenarios:
- Repo contribution: Place file in
clis/<site>/<name>.js,npm run buildto auto-register - Private adapter(local use, no PR needed): Place file in
~/.opencli/clis/<site>/<name>.js, no build needed
# Repo 贡献:build 后直接运行
npm run build
opencli list | grep mysite # 确认注册
opencli mysite mycommand --limit 3 -v # 实际运行
# 私人 adapter(~/.opencli/clis/):一键验证
opencli browser verify <site>/<name>
Done criteria: Command runs and returns a non-empty table, with fields matching expectations.
Step 5: Submit and publish
npm run build && opencli mysite mycommand --limit 3 # 最终验证(Repo 贡献场景)
git add clis/mysite/ && git commit -m "feat(mysite): add mycommand" && git push
Common pitfalls
| Pitfall | Symptom | Solution |
|---|---|---|
Missing navigate |
Target page context error |
Add before evaluate page.goto() |
Missing strategy: public |
Public API still launches browser | Add strategy: Strategy.PUBLIC + browser: false |
| Blocked by risk control (fake 200) | Core data in JSON is empty string | Must assert!throw new AuthRequiredError(domain) prompts re-login |
| SPA returns HTML | fetch('/api/xxx') returns <!DOCTYPE html> |
Page host is app.xxx.com, real API is at api.xxx.com; search JS bundle for baseURL |
| 400 missing context Header | Still 400 with Bearer, reports Missing X-Server-Id |
First call /servers to get business context ID, add to headers |
| File written to wrong directory | opencli list Command not found |
Put Repo contributions in clis/<site>/ + build; put private adapters in ~/.opencli/clis/<site>/ |
| TS evaluate format | () => {} reports result is not a function |
Must use IIFE:(async () => { ... })() |
| Large blocks of JS embedded in evaluate | String escaping issues, difficult to debug | Put logic in func() and write it in native TS |
| Page loads asynchronously | evaluate gets empty data | Use polling inside evaluate to wait, or increase the wait time |
| Cookie expired | Returns 401 / empty data | Log in to the target site again in the browser |
More references
| Documentation | Content |
|---|---|
| adapter-templates.md | Complete Tier 1~4 templates, pagination patterns, error handling conventions |
| advanced-patterns.md | Cascading requests, tap debugging, Verbose mode, anti-change mode |
| record-workflow.md | Manual recording approach (suitable for complex interactive pages) |
| opencli-oneshot skill | Single-point quick generation (only needs one URL + target description) |
Auto-generate with AI Agent
# 一键:探索 → 分析 → 合成 → 注册
opencli generate https://www.example.com --goal "hot"
# 或分步:
opencli explore https://www.example.com --site mysite
opencli synthesize mysite
opencli verify mysite/hot --smoke
The generated candidate TS is saved in .opencli/explore/mysite/candidates/, copy it to clis/mysite/ and fine-tune it.