SuperAPI AI Directory

opencli-explorer

Coding & development

View repository

Updated 10-15 22:40

About this skill

CLI-EXPLORER — The Complete Guide to Exploratory Adapter Development

From zero to release: API discovery → auth strategy → write adapter → test and verify.

Choose a path first

Situation Go here
You only need to generate one command for one specific page opencli-oneshot skill
You want to let the machine try automatically first opencli generate <url> [--goal <goal>], and come back if it fails
New site / multiple commands / oneshot stuck Keep reading this document
The output is going to be submitted as a PR This document + clis/<site>/ + npm run build
Just for local private use, no PR This document + ~/.opencli/clis/<site>/

Core workflow

 ┌─────────────┐     ┌─────────────┐     ┌──────────────┐     ┌────────┐
 │ 1. 发现 API  │ ──▶ │ 2. 选择策略  │ ──▶ │ 3. 写适配器   │ ──▶ │ 4. 测试 │
 └─────────────┘     └─────────────┘     └──────────────┘     └────────┘
   browser explore     cascade             TS cli() API         verify

Required reading for AI Agents: you must explore with a browser

[!CAUTION] You must open the target website in a browser to explore! Don't rely on static analysis alone. Many APIs arelazy-loaded—deep data such as subtitles, comments, and follow lists is only triggered after clicking.

Browser exploration workflow

Step Command What it does
0. Open the page opencli browser open <url> Navigate to the target page and start capturing
1. Observe elements opencli browser state View interactive elements (buttons/tabs), with [N] indexes
2. First capture opencli browser network List the captured JSON API requests
3. Simulate interaction opencli browser click <N> Click buttons to trigger lazy-loaded APIs
4. Second capture opencli browser network Find the newly triggered APIs
5. View the response opencli browser network --detail <N> View the full response body
6. Verify the API opencli browser eval "fetch(...).then(r=>r.json())" Confirm the API is reproducible

Common mistakes

❌ Wrong approach ✅ Correct approach
Only use opencli explore, and wait for the results Use opencli browser open to browse proactively
Write code directly without looking at browser requests First opencli browser network to confirm the API
Capture directly after opening the page Use opencli browser click to simulate interaction and trigger lazy loading
Give up when HTTP 200 but the data is empty Check whether signature or Cookie authentication is required (fake 200 risk control)
Rely on __INITIAL_STATE__ to get all the data __INITIAL_STATE__ only has first-screen data; deep data requires calling the API
opencli browser network is empty Re- open refresh and capture; or check whether the API is on a separate domain

Hands-on example: implement a "follow list" adapter in 5 minutes

opencli browser open https://space.bilibili.com/{uid}/fans/follow
opencli browser network
#   [0] GET 200 /x/relation/followings?vmid={uid}&pn=1&ps=24
opencli browser network --detail 0
# 确认数据结构:{ code: 0, data: { total: 1342, list: [{mid, uname, ...}] } }
opencli browser eval "fetch('/x/relation/followings?vmid=137702077&pn=1&ps=5', {credentials:'include'}).then(r=>r.json())"
# → 有数据,结论:Tier 2 Cookie,写 following.js

Step 1: Discover the API

Main path: proactive browser exploration

Use the workflow above to open the page → simulate interaction → capture. Pay attention to:

  • URL pattern:/api/v2/hot?limit=20 → the endpoint to call
  • Method:GET / POST
  • Request Headers: Cookie? Bearer? Custom signature headers (X-s, X-t)?
  • Response Body: data path (data.items、data.list)

Advanced shortcuts (try in order of priority)

  1. Suffix brute-force method (.json): Reddit, Xueqiu, etc., add .json to the URL to get REST data directly (Tier 2 instant win)
  2. Global state method (__INITIAL_STATE__): SSR sites (Bilibili, Xiaohongshu) have homepage data mounted on window
  3. Active interaction trigger method: Lazy-loaded APIs require clicking buttons ("CC", "Expand All") to trigger
  4. Framework Store truncation: Vue + Pinia sites, Store Actions complete signing for you
  5. XHR/Fetch interception: Last resort, use installInterceptor packet capture

Framework detection

opencli browser eval "(()=>{
  const vue3 = !!document.querySelector('#app')?.__vue_app__;
  const pinia = vue3 && !!document.querySelector('#app').__vue_app__.config.globalProperties.\$pinia;
  const react = !!window.__REACT_DEVTOOLS_GLOBAL_HOOK__;
  return JSON.stringify({vue3, pinia, react});
})()"

Vue + Pinia → Can use Store Actions to bypass signing (Tier 4).


Step 2: Choose authentication strategy

opencli cascade https://api.example.com/hot   # 自动探测

Strategy decision tree

fetch(url) 直接能拿到?
  → ✅ Tier 1: public(browser: false,~1s)
  → ❌ fetch(url, {credentials:'include'}) 带 Cookie 能拿到?
       → ✅ Tier 2: cookie(最常见)
       → ❌ localStorage 有 token,Bearer header 能拿到?
              → ✅ Tier 2.5: localStorage Bearer(现代 SaaS 主流)
                  带了 Bearer 但 400 "Missing X-Xxx header"?
                  → 先调 /servers 或 /workspaces 拿业务上下文 ID
              → ❌ 加 CSRF header 后能拿到?
                     → ✅ Tier 3: header(如 Twitter ct0 + Bearer)
                     → ❌ 网站有 Pinia/Vuex Store?
                            → ✅ Tier 4: intercept(Store Action + XHR 拦截)
                            → ❌ Tier 5: ui(UI 自动化,最后手段)

Strategy comparison

Tier Strategy Speed Applicable scenarios Examples
1 public ⚡ ~1s Public API, no login required Hacker News, V2EX
2 cookie 🔄 ~7s Cookie authentication is sufficient Bilibili, Zhihu, Reddit
2.5 localStorage Bearer 🔄 ~7s JWT stored in localStorage, API on separate domain Slock, Linear, Notion
3 header 🔄 ~7s Requires CSRF token or Bearer Twitter GraphQL
4 intercept 🔄 ~10s Requests have complex signing Xiaohongshu (Pinia + XHR)
5 ui 🐌 ~15s+ No API, pure DOM parsing Legacy websites

Step 2.5: Preparation

Reuse existing adapters first, don't start from scratch:

ls clis/<site>/             # 看同站点已有什么
cat clis/<site>/feed.js     # 读最相似的那个

Only 3 changes needed:name, API URL, field mapping.

Bilibili platform SDK(clis/bilibili/utils.js):

  • fetchJson(page, url) → Regular Cookie-tier API
  • apiGet(page, path, {signed, params}) → Interface with URL containing /wbi/ (auto-signed)
  • getSelfUid(page) / resolveUid(page, input) → User UID handling

Step 3: Write adapter

All adapters uniformly use the cli() API, place into clis/<site>/<name>.js to auto-register.

Complete templates (Tier 1~4), pagination patterns, error handling conventions (throw CliError + YAML envelope)→ adapter-templates.md

Minimal structure(Tier 2 Cookie):

import { cli, Strategy } from '@jackwener/opencli/registry';

cli({
  site: 'mysite',
  name: 'mycommand',
  description: '一句话描述',
  domain: 'www.example.com',
  strategy: Strategy.COOKIE,
  browser: true,
  args: [{ name: 'limit', type: 'int', default: 20 }],
  columns: ['rank', 'title', 'value'],
  func: async (page, kwargs) => {
    await page.goto('https://www.example.com');
    const data = await page.evaluate(`(async () => {
      const res = await fetch('/api/items', { credentials: 'include' });
      const d = await res.json();
      return d.data?.items || [];
    })()`);
    return (data as any[]).slice(0, kwargs.limit).map((item, i) => ({
      rank: i + 1,
      title: item.title || '',
      value: item.value || '',
    }));
  },
});

Cascading requests, tap debugging, anti-change patterns → advanced-patterns.md


Step 4: Test

Build passing ≠ Functioning correctly. Must actually run and confirm output.

Two development scenarios:

  • Repo contribution: Place file in clis/<site>/<name>.js,npm run build to auto-register
  • Private adapter(local use, no PR needed): Place file in ~/.opencli/clis/<site>/<name>.js, no build needed
# Repo 贡献:build 后直接运行
npm run build
opencli list | grep mysite                 # 确认注册
opencli mysite mycommand --limit 3 -v      # 实际运行

# 私人 adapter(~/.opencli/clis/):一键验证
opencli browser verify <site>/<name>

Done criteria: Command runs and returns a non-empty table, with fields matching expectations.


Step 5: Submit and publish

npm run build && opencli mysite mycommand --limit 3   # 最终验证(Repo 贡献场景)
git add clis/mysite/ && git commit -m "feat(mysite): add mycommand" && git push

Common pitfalls

Pitfall Symptom Solution
Missing navigate Target page context error Add before evaluate page.goto()
Missing strategy: public Public API still launches browser Add strategy: Strategy.PUBLIC + browser: false
Blocked by risk control (fake 200) Core data in JSON is empty string Must assert!throw new AuthRequiredError(domain) prompts re-login
SPA returns HTML fetch('/api/xxx') returns <!DOCTYPE html> Page host is app.xxx.com, real API is at api.xxx.com; search JS bundle for baseURL
400 missing context Header Still 400 with Bearer, reports Missing X-Server-Id First call /servers to get business context ID, add to headers
File written to wrong directory opencli list Command not found Put Repo contributions in clis/<site>/ + build; put private adapters in ~/.opencli/clis/<site>/
TS evaluate format () => {} reports result is not a function Must use IIFE:(async () => { ... })()
Large blocks of JS embedded in evaluate String escaping issues, difficult to debug Put logic in func() and write it in native TS
Page loads asynchronously evaluate gets empty data Use polling inside evaluate to wait, or increase the wait time
Cookie expired Returns 401 / empty data Log in to the target site again in the browser

More references

Documentation Content
adapter-templates.md Complete Tier 1~4 templates, pagination patterns, error handling conventions
advanced-patterns.md Cascading requests, tap debugging, Verbose mode, anti-change mode
record-workflow.md Manual recording approach (suitable for complex interactive pages)
opencli-oneshot skill Single-point quick generation (only needs one URL + target description)

Auto-generate with AI Agent

# 一键:探索 → 分析 → 合成 → 注册
opencli generate https://www.example.com --goal "hot"

# 或分步:
opencli explore https://www.example.com --site mysite
opencli synthesize mysite
opencli verify mysite/hot --smoke

The generated candidate TS is saved in .opencli/explore/mysite/candidates/, copy it to clis/mysite/ and fine-tune it.