What is the Tonggu Large Model
The Tonggu Large Model is an artificial intelligence language model launched by the Deep Learning and Vision Computing Lab (SCUT-DLVCLab) of South China University of Technology, focusing on the processing of ancient Chinese classical texts. It is based on Baichuan2-7B-Base for incremental pretraining, using 2.41 billion ancient book corpus for unsupervised training, combined with 4 million ancient book dialogue data for instruction fine-tuning. The model adopts redundancy-aware tuning (RAT) technology, effectively improving performance on ancient book tasks. It helps users understand and translate ancient book documents more conveniently. Through retrieval-augmented generation (CCU-RAG) technology, it reduces hallucination problems in knowledge-intensive tasks and improves the accuracy and reliability of generated content.
Main Functions of the Tonggu Large Model
- Ancient Chinese Punctuation: The Tonggu Large Model can automatically add punctuation marks to ancient Chinese texts, solving the common sentence segmentation problems in ancient books and helping users better understand the content of ancient Chinese texts.
- Classical-to-Vernacular Translation: The model supports bidirectional translation between classical Chinese and vernacular Chinese, translating obscure ancient Chinese into modern Chinese, and can also convert modern Chinese into classical Chinese, making it convenient for users to read and research ancient books.
- Poetry Creation: The Tonggu Large Model can generate poems that conform to the metrical patterns and styles of ancient poetry. Users can provide themes or keywords as needed, and the model generates corresponding poetry works.
- Ancient Book Appreciation: The model can appreciate and analyze classic chapters in ancient books, interpreting their literary value, historical background, and cultural connotations, assisting users in in-depth study of ancient books.
- Ancient Book Retrieval and Question Answering: Combined with retrieval-augmented technology, the Tonggu Large Model can quickly retrieve ancient book content and provide accurate answers based on users' questions, helping users efficiently obtain ancient book information.
- Assisting Ancient Book Collation: The model can identify text errors, omissions, and other problems in ancient books, provide repair suggestions, and assist in the collation and digitization of ancient books.
Technical Principles of the Tonggu Large Model
- Base Model Architecture: The Tonggu Large Model is based on Baichuan2-7B-Base for incremental pretraining. Baichuan2-7B-Base is a powerful pretrained language model that provides the Tonggu Large Model with foundational language understanding and generation capabilities.
- Unsupervised Incremental Pretraining: The model undergoes unsupervised incremental pretraining on a 2.41 billion ancient book corpus. This enables the model to learn the language style and structure of ancient books, laying the foundation for subsequent ancient book processing tasks.
- Multi-Stage Instruction Fine-Tuning: The Tonggu Large Model adopts multi-stage instruction fine-tuning technology and proposes the redundancy-aware tuning (RAT) method. While improving downstream task performance, it retains the capabilities of the base model. Through instruction fine-tuning, the model can better adapt to specific tasks in ancient book processing, such as ancient Chinese translation and punctuation.
- Retrieval-Augmented Generation (RAG) Technology: The Tonggu Large Model combines retrieval-augmented generation (RAG) technology to reduce hallucination problems in knowledge-intensive tasks. The core is to combine information retrieval with text generation, by retrieving relevant information from external knowledge bases and inputting it as context to the language model, generating more accurate and contextually appropriate answers.
Project Address of the Tonggu Large Model
- Github Repository：https://github.com/SCUT-DLVCLab/TongGu-LLM
- HuggingFace Model Library：https://huggingface.co/SCUT-DLVCLab/TongGu-7B-Instruct
Application Scenarios of the Tonggu Large Model
- Ancient Book Processing and Digitization: The Tonggu large model can efficiently process ancient books and documents, supporting functions such as classical-to-vernacular translation, punctuation and sentence segmentation, and ancient book retrieval. It assists in the collation of ancient books, improving the efficiency of digitizing ancient books through intelligent recognition and correction of textual errors in ancient books.
- Educational Support: Teachers can use it to generate lesson plans, teaching PPTs, and design classroom interactive activities. For students, the model can provide functions such as classical Chinese translation, idiom explanations, and poetry creation, helping them better understand ancient texts.
- Cultural Inheritance and Popularization: By reducing the difficulty of reading ancient books, the Tonggu large model allows more people to encounter and understand traditional Chinese culture.
- Academic Research: The Tonggu large model provides strong technical support for ancient book research, helping scholars quickly retrieve and analyze the content of ancient books.