citation-and-reference-validator

Citation & Reference Validator

Overview

This skill is designed to conduct a quality audit of citations (citations) and the bibliography (references) in a scientific manuscript. The agent ensures that all references have valid metadata (free from fake references resulting from AI hallucination) and that the text of reference sentences in the manuscript is actually factually supported by the cited paper (contextual alignment).

Dependencies

• literature-search-openalex
• pubmed-database

Quick Start

Example usage: "Use the citation-and-reference-validator skill to check whether the claim sentence 'Method X has an accuracy of 98% (Smith et al., 2022)' is actually supported by the content of the Smith et al. paper with DOI 10.1000/xyz."

Workflow

1. Reference Metadata Validation (Programmatic Metadata Integrity Audit)

• Programmatic Guardrail Execution (MANDATORY): Before performing manual assessment, the agent MUST execute the automated verification script to validate the bibliography against the Crossref and OpenAlex APIs. (Note: Determine the absolute path of validate_references.py based on the location of the directory SKILL.md is in, whether local or global installation):

  python "<PATH_KE_SKILL>/scripts/validate_references.py" --input references.json --output report.json
  

  or run directly for a single DOI/Title:

  python "<PATH_KE_SKILL>/scripts/validate_references.py" --doi <DOI> --title "<TITLE>"
  

• Evaluate the output of report.json:
  • VALID: Metadata officially verified by public databases (Crossref/OpenAlex).
  • METADATA_MISMATCH: The paper is found, but the given year/author/title attributes differ significantly from the original database.
  • HALLUCINATED: The DOI or title is not found in any public database. Must be immediately rejected / removed from the reference list.
• Mark references with the status HALLUCINATED or METADATA_MISMATCH explicitly in the audit report.

2. Citation Context Alignment Validation (Evidence-Based Citation Grounding)

• Programmatic Snippet Fetcher Execution (MANDATORY): To check whether the manuscript's claim is factually supported by the content of the original paper, the agent MUST execute the verbatim evidence-fetching script:

  python "<PATH_KE_SKILL>/scripts/fetch_evidence_snippet.py" --claim "<KALIMAT_KLAIM>" --doi "<DOI>"
  

• Evaluate the output evidence_snippets from the Python script:
  • FULLY_SUPPORTED: There is a verbatim quote in the original paper that aligns 100% with the manuscript's claim.
  • PARTIALLY_SUPPORTED: There is a relevant quote but it requires additional qualification/context in the manuscript (e.g., "only applies to dataset X").
  • CONTRADICTORY: The original paper's quote actually states the opposite of the manuscript's claim.
  • UNSUPPORTED / MISATTRIBUTED: The claim is not discussed or the paper discusses a completely different topic.

3. Citation Style & Consistency Check (Format Check)

• Ensure citation writing consistently follows a particular style guide (e.g., APA, IEEE, Harvard, MLA) both in the text and in the bibliography.
• Ensure that all documents cited in the text are listed in the bibliography, and vice versa.

4. Reference Validation Report Format

Present the reference audit report in the following mandatory evidence table format:

Manuscript Sentence / Claim Text	Reference Paper (Citation & DOI)	Original Verbatim Quote from the Paper (Evidence Snippet)	Contextual Support Status	Correction Notes

Common Mistakes & Critical Rules (Anti-Reference Hallucination)

• Hallucinated References (Fake References): Including AI-generated references that look convincing (featuring famous author names and plausible titles) but were never actually published or have fake DOIs.
• Misleading Chain Citation (Lazy Citation): Citing paper B that cites paper A for a theory, when the actual content of paper A differs from paper B's interpretation. The agent must trace the primary source if there is a possibility that the PDF/full text is openly accessible.
• Overgeneralization Citation: Citing a specific experimental paper as if it proves a universal theory without mentioning the experimental testing limitations of that paper.
• Citation Formatting Errors: Being inconsistent in compiling the reference list (randomly mixing APA and IEEE formats).
• API Key Obstacles: Stopping the audit process or showing an error message when the Semantic Scholar API Key is not filled in. Perform cross-validation using free public APIs (OpenAlex / Crossref) that do not restrict access without a token.