SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsreference-verify

reference-verify

Writing & research

View repository

Updated 10-15 17:30

About this skill

Ref-Verify: Academic Paper Reference Verification

Verify references for: $ARGUMENTS

Purpose

This skill performs a systematic audit of every reference cited in a LaTeX manuscript. For each citation it:

  1. Confirms the BibTeX entry describes a real, published paper (not hallucinated).
  2. Locates every place the reference is cited in the manuscript.
  3. Extracts the claim or viewpoint the author attributes to that reference.
  4. Judges whether the cited paper's actual content supports the attributed claim.
  5. Produces a structured verification report as a LaTeX table and a Markdown summary.

Constants

  • VERIFY_METHOD = web — How to verify references. Options: web (WebSearch + WebFetch for each entry), dblp (DBLP API first, WebSearch fallback), manual (skip online verification, only check internal consistency).
  • OUTPUT_FORMAT = excel — Output format. Options: excel (default, .xlsx with conditional formatting), markdown, latex, all (all three).
  • OUTPUT_DIR — Directory to save the report. Defaults to the same directory as the input .tex file.
  • LANGUAGE = zh — Report language. Options: zh (Chinese headers), en (English headers).
  • STRICT = false — When true, flag any citation where the attributed claim cannot be confirmed from the paper's abstract/title as 未确认. When false, give benefit of the doubt for plausible matches.
  • APPLY_FIXES = interactive — Whether to apply the suggested fixes after the report is generated. Options: interactive (default — show each proposed edit and ask user y/n/skip/edit), auto (apply every fix without asking; use only when user explicitly opts in), off (stop after the report; no edits to source files).

Inputs

The skill accepts one of:

  1. A directory containing .tex and .bib files (scans all .tex files for citations).
  2. A single .tex file (uses the \bibliography{} command to locate the .bib file).
  3. A .bib file alone (verifies entries exist but cannot check in-text citation context).

If no argument is provided, scan the current working directory.

Workflow

Step 1: Locate Files

  1. Find all .tex files in the target directory (or the single specified file).
  2. Find the .bib file(s) referenced by \bibliography{} or \addbibresource{}.
  3. Read and parse the .bib file to extract all BibTeX entries.
  4. Read all .tex files to extract all \cite{}, \citep{}, \citet{}, \citealt{}, \citealp{} commands.

Step 2: Build Citation Map

For each citation key found in the .tex files:

  1. Extract BibTeX metadata: author, title, journal, year, volume, pages.
  2. Locate all in-text occurrences: Record each file name, line number, and the surrounding sentence (±1 sentence for context).
  3. Extract attributed claim: From the surrounding text, identify what viewpoint, finding, or argument the author attributes to the cited work.

Produce an internal data structure:

[
  {
    "seq": 1,
    "key": "baker2006investor",
    "bib": { "author": "Baker, Malcolm and Wurgler, Jeffrey", "title": "Investor sentiment...", "journal": "Journal of Finance", "year": "2006" },
    "occurrences": [
      { "file": "sections/1_introduction.tex", "line": 12, "context": "...investor sentiment predicts cross-sectional stock returns...", "claim": "investor sentiment predicts cross-sectional returns" }
    ]
  },
  ...
]

Step 3: Verify Each Reference

For each BibTeX entry, perform verification in order:

3a. Existence Check (Is the paper real?)

Use this verification chain (stop at first success):

Method A: DBLP (if VERIFY_METHOD includes dblp)

# Search DBLP by title + first author
curl -s "https://dblp.org/search/publ/api?q=TITLE+FIRST_AUTHOR&format=json&h=3"
# If match found: VERIFIED

Method B: WebSearch

WebSearch: "EXACT_TITLE" author FIRST_AUTHOR_LASTNAME journal JOURNAL_NAME

Check the top 3 results. A paper is VERIFIED if:

  • The title matches (exact or near-exact, allowing minor capitalization/punctuation differences).
  • The author(s) match.
  • The journal/venue matches.
  • The year matches (±1 year tolerance for working paper → publication lag).

A paper is UNVERIFIED if no matching result is found after both methods.

Method C: CrossRef DOI (if a DOI is in the bib entry)

WebFetch: https://doi.org/DOI

3b. Metadata Accuracy Check

If the paper is verified, compare the BibTeX metadata against the verified source:

  • Is the year correct?
  • Is the journal name correct?
  • Are the volume and pages correct?
  • Are all authors listed correctly?

Flag any discrepancies.

3c. Citation Context Check (Is the claim supported?)

For each occurrence of the citation in the manuscript:

  1. Read the sentence(s) surrounding the citation.
  2. Identify the specific claim attributed to the cited paper.
  3. Determine whether the cited paper's title, abstract, and known content support that claim.

Classification:

  • Supported: The claim is clearly consistent with the cited paper's known content.
  • Partially supported: The claim is related but overstated, narrowed, or slightly reframed.
  • Not supported: The claim does not match the cited paper's content.
  • Unconfirmed: Cannot determine from available information (title/abstract only).
  • Paper not found: The BibTeX entry could not be verified as a real publication.

Step 4: Generate Report

The default output is an Excel report (ref_verify_report.xlsx). Use Python + openpyxl to generate.

Excel Report (ref_verify_report.xlsx) — DEFAULT

The Excel workbook contains three sheets, structured as follows:

Sheet 1: Overview

Item Value
Paper Title [paper title]
Verification Date [date]
Verification Method [web/dblp/manual]
Total Citations X
Verification Passed Y
Questionable Z
Not Found W
Uncited Entries U

Sheet 2: Verified References

This sheet covers references that can be found online. Focus is on citation–claim support analysis.

Column Description
No. In order of first appearance in the text
Reference Author (Year), Journal Abbreviation
BibTeX Key cite key
Citation Location in Text Filename:line number, separate multiple with semicolons
Cited Claim The core claim expressed by the author at the citation location (≤30 characters)
Support Verdict Supported / Partially supported / Not supported / Unconfirmed
Support Analysis Detailed explanation: what the original text actually says vs. what the citation location claims, and where the difference lies
Metadata Issues Metadata errors such as year, journal name, author spelling, etc. (leave blank if none)
Revision Suggestions Specific revision suggestions for not supported/partially supported/metadata errors

Conditional formatting for this sheet:

  • Supported → row background light green (#C6EFCE)
  • Partially supported → row background light yellow (#FFEB9C)
  • Not supported → row background light red (#FFC7CE)
  • Unconfirmed → row background light gray (#D9D9D9)

Sheet 3: Problematic References

This sheet covers references that cannot be found or have serious issues. Each entry gets a detailed problem diagnosis.

Column Description
No. In order of first appearance in the text
Reference Author (Year), Journal in BibTeX
BibTeX Key cite key
Citation Location in Text Filename:line number
Cited Claim The core claim expressed by the author at the citation location
Issue Type See categories below
Issue Details Detailed description of the search process and findings
Possible Cause Analyze the reason why this entry has a problem
Revision Suggestions Specific revision plan (replace reference/correct information/delete citation)
Severity 🔴 High / 🟡 Medium / 🟢 Low

Issue Type Categories (fill in "Issue Type" column):

  • Paper not found: No matching result can be found; may be a fabricated reference
  • Author mismatch: A paper with the same title is found but the authors are different
  • Journal/Year mismatch: The paper exists but the publication journal or year does not match the bib entry
  • Title error: A similar paper is found but the title has obvious differences
  • Suspected confusion: Multiple similar papers; cannot determine which one is cited
  • Preprint/Unpublished: found only on arXiv/SSRN etc., not formally published
  • Orphan entry: exists in the bib but never cited in the main text

Conditional formatting for this sheet:

  • Severity 🔴 High → entire row bold red background (#FFC7CE), bold font
  • Severity 🟡 Medium → row background light orange (#FFD9B3)
  • Severity 🟢 Low → row background light blue (#BDD7EE)

Excel Generation Code Pattern

Use this Python pattern via Bash to generate the xlsx:

import openpyxl
from openpyxl.styles import Font, PatternFill, Alignment, Border, Side
from openpyxl.utils import get_column_letter

wb = openpyxl.Workbook()

# --- Sheet 1: Overview ---
ws1 = wb.active
ws1.title = "总览"
# Write overview stats...

# --- Sheet 2: Verified References ---
ws2 = wb.create_sheet("已验证文献")
headers2 = ["序号", "参考文献", "BibTeX Key", "文中引用位置", "引用观点",
            "支撑判定", "支撑分析", "元数据问题", "修改建议"]
# Write headers with bold + background
# Write rows, apply conditional fill per 支撑判定 value
# Auto-adjust column widths

# --- Sheet 3: Problematic References ---
ws3 = wb.create_sheet("问题文献")
headers3 = ["序号", "参考文献", "BibTeX Key", "文中引用位置", "引用观点",
            "问题类型", "问题详情", "可能原因", "修改建议", "严重程度"]
# Write headers, rows, apply severity-based fills
# Auto-adjust column widths

# Global styling
for ws in [ws1, ws2, ws3]:
    ws.sheet_properties.pageSetUpPr = openpyxl.worksheet.properties.PageSetupProperties(fitToPage=True)
    ws.freeze_panes = "A2"  # Freeze header row

wb.save("OUTPUT_DIR/ref_verify_report.xlsx")

Important: Before running, ensure openpyxl is installed: pip install openpyxl

Markdown Report (ref_verify_report.md) — only when OUTPUT_FORMAT includes markdown or all

# 参考文献验证报告

**论文**: [paper title]
**验证日期**: [date]
**验证方法**: [web/dblp/manual]
**总引用数**: X | **验证通过**: Y | **存疑**: Z | **不存在**: W

## 一、已验证文献

| 序号 | 参考文献 | 文中引用位置 | 引用观点 | 支撑判定 | 支撑分析 | 修改建议 |
|------|----------|-------------|----------|----------|----------|----------|
| 1 | Baker and Wurgler (2006), JF | 1_intro.tex:L12 | investor sentiment predicts returns | ✅ 支撑 | 与原文主要发现一致 | — |

## 二、问题文献

| 序号 | 参考文献 | 问题类型 | 问题详情 | 可能原因 | 修改建议 | 严重程度 |
|------|----------|----------|----------|----------|----------|----------|
| 5 | Smith (2020), AER | 论文不存在 | 搜索Google Scholar/DBLP均无结果 | 可能为AI生成的虚构文献 | 删除该引用或替换为... | 🔴 高 |

## 三、未引用条目
...

## 四、总结与建议
...

LaTeX Report (ref_verify_report.tex) — only when OUTPUT_FORMAT includes latex or all

Generate a LaTeX longtable with the same two-section structure (已验证 + 问题文献).

Step 5: Summary and Recommendations

After the table, provide:

  1. Overall assessment: How many references pass verification, how many are flagged.
  2. Critical issues: Any references that appear fabricated or seriously misattributed.
  3. Minor issues: Metadata errors (wrong year, wrong journal name, missing pages).
  4. Recommendations: Specific fixes for each flagged entry.

Step 6: Apply Recommended Modifications

The verification report is only useful if its "修改建议" column actually reaches the source files. Step 6 walks every actionable suggestion back into the .tex and .bib files. Skip this step only when APPLY_FIXES = off.

Why this matters. Authors typically have 30–100 citations. Hand-applying every suggested fix from the report is tedious and error-prone — they will silently skip the "small" ones (a wrong year, an orphan entry) and the manuscript ships with the same defects. Closing the loop here is the difference between a report and a real audit.

6a. Build the fix queue

Re-use the in-memory verification data from Steps 2–3 — do not re-parse the xlsx. Build an ordered queue of rows that need action:

  1. All rows from Sheet 3 (问题文献) — every entry, regardless of severity.
  2. Rows from Sheet 2 (已验证文献) where 支撑判定 = 部分支撑 — the claim is overstated, narrowed, or reframed.

Rows where 支撑判定 ∈ {支撑, 未确认} are not in the queue. 不支撑 rows belong on Sheet 3 already (move them there if the report put them on Sheet 2). Process the queue in 序号 order so the user sees fixes flow in the same order as the manuscript.

6b. For each queued row, propose a concrete edit

The "修改建议" column says what to do at the policy level (e.g. "替换为 Baker and Wurgler (2007)"). Step 6 turns it into an exact diff. Map by category:

Category Handling method Where to change
Partial support (Sheet 2) Rewrite the sentence containing the citation so the statement matches the actual conclusion of the original text; or replace it with a more appropriate cite key .tex
No support Same as above, or delete the citation .tex (+ .bib if cite key is also dropped)
Paper does not exist Prefer replacing with the substitute literature nominated in "修改建议"; if there is no substitute, delete the citation and rewrite the sentence so it stands on its own .bib (add/delete) + .tex (replace key or rewrite sentence)
Author mismatch / journal-year mismatch / title error Correct the BibTeX fields (author / journal / year / title / volume / pages) .bib
Suspected confusion Replace with the cite key of the correct paper, adding a bib entry if necessary .bib + .tex
Preprint/Unpublished If formally published → replace with the formal version; otherwise change the bib entry type to @unpublished or @misc and add note = {Working paper} .bib
Orphan entry Delete directly from .bib (low risk by default, can be done in batch) .bib

For each row, before showing the user, prepare:

  1. Target file & line: the exact .tex line for in-text edits; the BibTeX entry block for .bib edits.
  2. old_string: enough surrounding context to be unique. For sentence rewrites, include the full sentence; for bib edits, include the relevant field line(s).
  3. new_string: the proposed edit applied.
  4. One-line rationale: tied back to the 修改建议 + 原文实际内容.

6c. Walk the queue interactively (default)

When APPLY_FIXES = interactive, present each fix to the user using this exact template, then act on the response:

[修订 N/总数]  序号 K · <BibTeX Key>  ·  <类别>
位置: <file>:<line>     严重程度: <🔴/🟡/🟢/—>

引用观点: <引用观点 from report>
原文实际: <one sentence on what the cited paper actually says, or "—" for bib-only fixes>
修改理由: <one-line rationale>

— 当前 —
<old_string>

— 修改后 —
<new_string>

应用此修改？(y=apply / n=skip / e=let me edit new_string / q=quit Step 6)

Response handling:

  • y: call the Edit tool with the prepared old_string / new_string. Mark the row applied.
  • n: log as skipped with the user's reason if given. Move on.
  • e: ask the user for a replacement new_string, then re-confirm before applying.
  • q: stop the queue. Summarize what's been applied so far. Do not undo prior edits.

When APPLY_FIXES = auto, skip the prompt and apply every prepared edit, but still display the same block so the user can scroll back and audit. Auto mode requires explicit user opt-in (-- apply: auto); never silently auto-apply.

6d. Safety rails

These are non-negotiable — getting a citation fix wrong can introduce plagiarism risk or hallucinated content into a published paper.

  1. Backup before first edit. On the first applied edit of the session, copy each .tex and .bib file that will be touched to <file>.bak-YYYYMMDD-HHMMSS. One backup per file, not per edit.
  2. Never invent a cite key or paper. A replacement cite key must either (a) already exist in the .bib file, or (b) point to a paper you actually verified online during Step 3 — in which case generate a clean BibTeX entry from the verified metadata, do not paraphrase.
  3. Never delete a sentence whole if doing so leaves the paragraph ungrammatical or removes a substantive claim. If the sentence's only purpose was to introduce the bad citation, deletion is fine; otherwise rewrite so the surrounding logic still holds.
  4. Multi-occurrence citations: if the same cite key appears at multiple .tex locations and the fix is bib-side (metadata correction), one .bib edit covers all sites. If the fix is in-text (claim rewrite), each occurrence is its own queue row and gets reviewed separately — the same paper can be fine in one place and overstated in another.
  5. Preserve LaTeX command shape. \citep{key}, \citet{key}, \cite[p.~12]{key} are not interchangeable — when replacing a cite key, keep the same command and any optional arguments. When rewriting a sentence, keep the citation command in a sensible position.
  6. Re-run a quick post-check. After the queue is processed, re-grep the manuscript for every cite key that was touched. Confirm: bib entry exists for each remaining key; no stray empty \citep{} was left behind; no .bak filenames leaked into the manuscript.

6e. Write a change log

After Step 6 completes, write ref_verify_changes.md next to the report. One entry per applied/skipped row:

# 参考文献修订日志

**生成时间**: 2026-05-08 07:35
**总数**: 已应用 N · 跳过 M · 用户编辑 K · 失败 F

## 已应用

### 1. baker2006investor — 部分支撑
- 位置: `1_introduction.tex:42`
- 原: investor sentiment predicts cross-sectional stock returns in all market conditions
- 改: investor sentiment predicts cross-sectional stock returns more strongly among stocks that are difficult to value or arbitrage (Baker and Wurgler, 2006)
- 理由: 原文结论限定在难以估值/套利的股票，原句过度泛化

### 2. smith2020aer — 论文不存在
- 位置: `2_literature.tex:88`，`bib/refs.bib:142`
- 操作: 删除 `.bib` 条目；将文中 `\citep{smith2020aer}` 替换为 `\citep{shleifer1997limits}`
- 理由: 在 Google Scholar / DBLP / SSRN 均搜索不到 Smith (2020, AER)，疑似虚构；改用主张相近的 Shleifer & Vishny (1997)

## 跳过
...

This log is the audit trail — if a co-author later asks "what did the verifier change?", the answer is one file. It also lets the user sanity-check before committing.

Column Definitions

Sheet 2 — 已验证文献

Column name Meaning Filling rule
序号 Numbered in order of first appearance in the text Increment starting from 1
参考文献 Author (year), journal abbreviation e.g. "Baker and Wurgler (2006), JF"
BibTeX Key citation key e.g. "baker2006investor"
Citation location in text File name:line number e.g. "1_intro.tex:L42"; separate multiple citations with semicolons
Cited viewpoint The core viewpoint expressed by the author at the citation Summarize in one sentence, no more than 30 characters
Support judgment Whether the cited paper supports the viewpoint Support / Partial support / No support / Unconfirmed
Support Analysis Comparative Analysis Actual conclusion in the original text vs. what is claimed at the citation point; point out the discrepancies
Metadata Issues Incorrect bib entry information Year/journal/author spelling, etc.; leave blank if none
Revision Suggestions Specific revision plan For unsupported/partially supported/metadata errors

Sheet 3 — Problematic References

Column Name Meaning Filling Rules
No. Numbered in order of first appearance in the text Increment starting from 1
Reference Author (Year), Journal in BibTeX Excerpt bib information as-is
BibTeX Key citation key e.g. "smith2020aer"
Citation Location in Text Filename:Line Number Separate multiple citations with semicolons
Cited Claim The author's core claim at the citation point ≤30 characters
Problem Type Category Label Paper does not exist/author mismatch/journal-year mismatch/title error/suspected confusion/preprint unpublished/orphan entry
Problem Details Search process and findings What was searched, what was found, where it doesn't match
Possible Cause Analysis of the cause of the problem e.g. AI-generated, memory confusion, version differences, etc.
Revision Suggestions Specific fix plan Which paper to replace with/which fields to correct/delete the citation
Severity Risk Level 🔴 High (fabricated/serious misattribution) / 🟡 Medium (metadata error/ambiguous match) / 🟢 Low (orphan entry/preprint)

Key Rules

  1. Verify every entry — Do not skip any citation. Every \cite in the manuscript must appear in the report.
  2. No guessing — If you cannot confirm a paper's existence after web search, mark it as 未确认, not 支撑.
  3. Context matters — The same paper may be cited multiple times for different claims. Each occurrence is a separate row.
  4. Be specific — In the "Cited Claim" column, extract the actual claim, not a generic description like "discusses sentiment".
  5. Suggest fixes — For every flagged entry, provide a concrete fix (correct year, correct journal, alternative citation, etc.).
  6. Preserve order — Number entries by first appearance in the manuscript, not alphabetically.
  7. Handle multi-cite — For \citep{a, b, c}, create separate rows for each key but note they appear in the same citation group.
  8. Check both directions — Also flag BibTeX entries that exist in .bib but are never cited in any .tex file (orphan entries).
  9. Report uncited entries — List any .bib entries not referenced in the text in a separate "Uncited Entries" section (in Excel: added to Sheet 3 with 问题类型 = "Orphan Entry").
  10. Excel first — Always generate the .xlsx report. Only generate .md / .tex if OUTPUT_FORMAT is markdown, latex, or all.
  11. Two-group structure — Clearly separate verified references (Sheet 2) from problematic ones (Sheet 3). The two sheets serve different purposes: Sheet 2 focuses on citation–claim alignment analysis; Sheet 3 focuses on problem diagnosis and severity.
  12. Severity assessment — For every entry in Sheet 3, assign a severity level: 🔴 High (paper likely doesn't exist or is seriously misattributed), 🟡 Medium (metadata errors or ambiguous match), 🟢 Low (minor issues like orphan entries or preprint status).
  13. Rich problem diagnosis — For unfound references, document: what you searched for, what you found instead, possible reasons for the discrepancy, and an actionable fix. Do NOT just say "未找到" — explain WHY.
  14. Close the loop with Step 6 — A report nobody acts on is wasted work. Always proceed into Step 6 unless APPLY_FIXES = off or the user explicitly declines. The fix queue is all of Sheet 3 plus Sheet 2 rows where 支撑判定 = 部分支撑 — process in 序号 order so the user can follow along with the manuscript.
  15. Backup, then edit — Step 6 makes destructive changes to .tex and .bib. Always create timestamped backups on the first applied edit, and never invent a replacement cite key without an actual verified source behind it. Write ref_verify_changes.md so the user has an audit trail.

Invoking

/ref-verify                                → scan current directory, output Excel
/ref-verify "paper/"                       → scan paper/ directory, output Excel
/ref-verify "main.tex"                     → scan specific file, output Excel
/ref-verify "paper/" -- format: all        → output Excel + Markdown + LaTeX
/ref-verify "paper/" -- format: markdown   → output Markdown only
/ref-verify "paper/" -- strict: true       → strict mode (flag all unconfirmed)
/ref-verify "paper/" -- method: dblp       → use DBLP API first
/ref-verify "paper/" -- lang: en           → English report headers
/ref-verify "paper/" -- apply: interactive → review & apply each fix one by one (default)
/ref-verify "paper/" -- apply: auto        → apply every suggested fix without asking
/ref-verify "paper/" -- apply: off         → stop after report; do NOT touch .tex / .bib

Related Skills

  • /econ-paper-write — Write the paper sections that need citation verification
  • /research-lit — Find replacement papers if a citation is invalid
  • /paper-compile — Compile the paper after fixing references
  • /arxiv — Search arXiv for preprint versions of cited papers