KB Abstract Fetch
Core Goal
- Reuse the same PostgreSQL connection env variables as
kb-meta-fetch.
- Select rows whose
abstract is empty and order by newest created_at first.
- Open
https://doi.org/<doi> in OpenClaw Browser and extract abstract text.
- Write back only when the row is still empty at update time.
- Default to dry run; require explicit
--apply to write.
Required Environment
KB_DB_HOST
KB_DB_PORT
KB_DB_NAME
KB_DB_USER
KB_DB_PASSWORD
KB_LOG_DIR (required run log directory)
Workflow
- Run local self-test first (no DB/browser required):
python3 scripts/kb_abstract_fetch.py --self-test
- Dry run first (default mode; no DB write):
python3 scripts/kb_abstract_fetch.py --limit 100
- Apply updates after review:
python3 scripts/kb_abstract_fetch.py --limit 100 --apply
- Override table/column names when needed (
created_at is fixed and required):
python3 scripts/kb_abstract_fetch.py \
--table journals \
--doi-column doi \
--abstract-column abstract \
--limit 100 \
--apply
Safety Contract
- Selection filter:
- DOI not empty
abstract empty (NULL or blank)
- Selection order:
- newest
created_at first (ORDER BY created_at DESC NULLS LAST LIMIT n)
- Update filter (second guard):
WHERE doi = ? AND abstract is still empty
- Run summary:
- emit
RUN_SUMMARY_JSON=<json> for current run only.
- Abort behavior:
- stop early when errors exceed
--max-errors.
Browser Requirement
openclaw CLI must be installed.
- Script checks
openclaw browser status; if browser is not running, it tries openclaw browser start.
- If start fails (for example extension tab not attached), attach OpenClaw browser session first, then rerun.
Script
scripts/kb_abstract_fetch.py