What is LongCite
LongCite is a project launched by Tsinghua University, aiming to improve the credibility and verifiability of large language models (LLMs) in long-text question-answering tasks. The project enables users to verify whether the model's answers are accurate by generating fine-grained sentence-level citations. Its core components include the LongBench-Cite evaluation benchmark, the CoF automated data construction pipeline, the LongCite-45k dataset, and the LongCite-8B and LongCite-9B models trained on this dataset. The models can understand long-text content and provide accurate question-answering services, while attaching text citations for direct reference, enhancing the transparency and reliability of information.
Main features of LongCite
- Generate fine-grained citations: LongCite enables language models to generate citations precise to the sentence level when answering long-text questions, allowing users to directly trace specific information in the original text.
- Improve answer faithfulness: LongCite helps ensure that the model's answers are more faithful to the original text and reduces the model's "hallucinations" (i.e., generating information inconsistent with the original text).
- Enhance verifiability: Users verify the truthfulness and accuracy of answers based on the fine-grained citations provided by the model, improving the credibility of the model's output.
- Automated data construction: LongCite adopts the CoF (Coarse to Fine) pipeline to automatically generate high-quality long-text question-answering data with fine-grained citations, providing rich annotated resources for model training.
- Evaluation benchmark: LongCite introduces the LongBench-Cite evaluation benchmark to measure the model's ability to generate citations in long-text question answering, including correctness and citation quality.
Technical principles of LongCite
- Long-text processing capability: LongCite supports large language models with ultra-long context windows (such as GLM-4-9B-1M, Gemini 1.5, etc.), and can process and understand texts tens of thousands of characters long.
- Fine-grained citation generation: LongCite trains models to generate citations precise to the sentence level, so that each answer can be traced to specific sentences in the original text, improving the verifiability of answers.
- Automated data construction pipeline (CoF): Uses the Self-Instruct method to automatically generate question-answer pairs from long texts. It retrieves sentence chunks related to the answer from the long text and generates chunk-level citations. Based on the chunk-level citations, it extracts the specific sentences supporting each statement and generates sentence-level citations.
- Supervised Fine-Tuning (SFT): The high-quality dataset with fine-grained citations generated by the CoF pipeline is used to fine-tune large language models, improving model performance in long-text question-answering tasks.
Project address of LongCite
- GitHub repository：https://github.com/THUDM/LongCite
- HuggingFace model library：https://huggingface.co/THUDM
- arXiv technical paper：https://arxiv.org/pdf/2409.02897
Application scenarios of LongCite
- Academic research: Researchers and scholars use LongCite to query large amounts of literature and obtain detailed answers with citations, supporting research work.
- Legal consultation: Legal professionals use LongCite to analyze legal documents and obtain specific legal provisions or case citations, supporting legal analysis and case research.
- Financial analysis: Financial analysts and investors use LongCite to understand complex financial reports and market research, and obtain accurate citations of key data and trends.
- Medical Consultation: Medical professionals rely on LongCite to query medical literature and obtain citations for diagnostic and treatment recommendations based on the latest research findings.
- News Reporting: Journalists and news organizations use LongCite to verify information in reports, ensuring that published news content is accurate and providing reliable source citations.