Purpose
"Large Language Models" is a Chinese technical book for beginners, intended to systematically explain the overall framework and roadmap of large model technology, provide Chinese reference materials for readers, and help popularize and disseminate the latest progress and technical system of large model technology.
Main Capabilities
This book is suitable for readers with a foundation in deep learning and can serve as a basic reference book on large models. The content covers many foundational topics such as pretraining, fine-tuning, alignment, and prompt engineering, and is organized and explained based on the editors' long-term research experience in large model technology and first-hand experience developing models such as Wenlan and Yulan. Each chapter is handled by the team according to division of labor, as follows: Chapter 3 leads: Min Yingqian, Yang Chen; participants: Li Junyi, Zhou Kun; Chapter 4 leads: Zhang Junjie, Hou Yupeng, Zhou Kun; Chapter 5 lead: Dong Zican; participants: Tian Zhen, Tang Tianyi; Chapter 6 leads: Tang Tianyi, Chen Yushuo; Chapter 7 lead: Tang Tianyi; participant: Cheng Xiaoxue; Chapter 8 leads: Li Junyi, Chen Zhipeng; Chapter 9 leads: Chen Yushuo, Liu Peiyu, Tang Tianyi; participant: Zhou Kun; Chapter 10 leads: Li Junyi, Tang Xinyu, Du Yifan; participant: Wang Xiaolei; Chapter 11 leads: Ren Ruiyang, Jiang Jinhao; participant: Li Junyi; Chapter 12 leads: Zhang Beichen, Zhou Kun; participant: Zhang Gaowei; Chapter 13 lead: Zhou Kun; participants: Jiang Jinhao, Li Yifan, Liu Zikang, Sun Wenqi, Wang Yuhao, Xu Lanling, Yang Jinxia, Zheng Bowen.
How to Use
This book can be used as a reference book for in-depth study of large model technology. Accompanying resources include:
- PDF course teaching slides: Courses that list this printed book as the main textbook can use an institutional email to contact rucaibox at 163.com to apply for the original PPT slides. The email subject format is: "Large Language Models" PPT slides-{Name}-{Institution}. Secondary distribution and online distribution are prohibited without permission. No reply will be given to emails that lack a basic introduction to the individual and course, copy template content, do not use an email authenticated by the sender's institution, or are for corporate training and personal study purposes (corporate needs must go through a commercial process; individuals can directly use the PDF version).
- Code snippets: All code snippets in this book have been made public on GitHub (original link: code).
- Teaching videos: Related course teaching videos have been made public on the bilibili website (original link: watch videos).
- Electronic version: A download link for the electronic version of the previous Chinese book is provided (download link 1, download link 2).
- Feedback channels: Opinions, comments, and suggestions can be fed back through the GitHub Issues page, or sent by email to the authors' email addresses batmanfly at qq.com, lijunyi at ruc.edu.cn, francis_kun_zhou at ruc.edu.cn, steventianyitang at outlook.com.
Applicable Scenarios
Suitable for readers with a foundation in deep learning to study large model technology. It can be used as a textbook or reference book for related university courses, and can also serve as a systematic learning reference for related practitioners.
Limitations
This book is positioned as an overall technical explanation for beginners, not a simple translation of an English survey; the content has undergone extensive updates and reorganization. Readers need to have a foundation in deep learning. The accompanying PDF courseware is only available upon application for courses that list this book as the primary textbook, and it must not be redistributed or disseminated online; dissemination without permission is prohibited. Courseware applications for corporate training and personal learning purposes will not receive a reply (individuals can directly use the PDF version for personal learning). The large model application chapter has not been included in the official publication; all content is subject to the official publication. The writing team plans to continuously update and improve the content online and welcomes readers' criticism and suggestions.
Background and Sources
The writing team published the English survey "A Survey of Large Language Models" in March 2023 and has continuously updated and improved it, now updated to the 14th version, with 95 pages of main text and 1,064 references. In August 2023, the Chinese translation of this survey (v10) was released. At the end of December 2023, the writing of the Chinese book was initiated, the first draft was completed in April 2024, and after 5 months of revision and improvement, it was published. This book embodies the core development experience of the "Yulan" project team. While providing mainstream PyTorch implementations, the Yulan project also completed synchronous support for the PaddlePaddle framework (PaddlePaddle version), ensuring the generality and autonomy of the technology. The writing and publication of this book received project support from the Beijing Science and Technology Program.
Recommendations
Zhang Hongjiang (Director of the Academic Advisory Committee of the Beijing Academy of Artificial Intelligence, Foreign Member of the National Academy of Engineering of the United States) commented that the content of this book is deeply combined with the editors' first-hand experience and comprehensively covers knowledge in many aspects of large model technology, and strongly recommends reading it. E Weinan (Chair Professor at Peking University, Academician of the Chinese Academy of Sciences) commented that this book has important reference and learning value. Zhang Yaqin (Chair Professor of Intelligent Science at Tsinghua University, Foreign Member of the Chinese Academy of Engineering) commented that this book covers many foundational topics such as pretraining, fine-tuning, alignment, and prompt engineering, and can provide authoritative and systematic learning reference for related practitioners.
