LLMsPracticalGuide is a carefully curated resource list of practical guides for large language models (LLMs), built based on the paper "Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond," with contributions from @xinyadu. This repository is continuously updated and aims to help practitioners navigate the vast landscape of large language models and their use in natural language processing (NLP) applications.
Main purposes:
- Provide an evolutionary tree of LLMs, tracing the development of language models in recent years and highlighting some of the most well-known models.
- Organize practical guide resources, including models, data, NLP tasks, prompt engineering, alignment, usage restrictions, and more.
- Include usage restrictions based on model and data license information.
- Provide a decision flowchart to help users assess whether their downstream NLP application meets specific conditions, thereby determining whether an LLM or a fine-tuned model is more suitable.
Main capabilities:
- Model guide: Covers BERT-style (Encoder-Decoder or Encoder-only) and GPT-style (Decoder-only) language models, listing specific models such as BERT, RoBERTa, DistilBERT, ALBERT, UniLM, ELECTRA, T5, GLM, AlexaTM, ST-MoE, GPT, GPT-2, GPT-3, OPT, PaLM, BLOOM, MT-NLG, GLaM, Gopher, chinchilla, LaMDA, LLaMA, GPT-4, BloombergGPT, GPT-NeoX-20B, PaLM 2, LLaMA 2, Claude 2, etc., with links to papers.
- Data guide: Includes pretraining data, fine-tuning data, and test/user data, involving datasets such as RedPajama, The Pile, and related papers.
- NLP task guide: Covers traditional NLU tasks, generation tasks, knowledge-intensive tasks, scaling capabilities, specific tasks, real-world tasks, efficiency, trustworthiness, benchmarking, instruction tuning, alignment (safe, harmless, truthful, honest), prompt engineering (helpful), open-source community alignment efforts, and more.
- Usage restrictions: Provides usage restrictions for models and data (commercial and research use).
- Other practical guides: Links to related blogs and repositories, such as "Why did all of the public reproduction of GPT-3 fail?", "Building LLM applications for production", "Data-centric Artificial Intelligence", etc.
How to use:
- Users can directly visit the GitHub repository to browse the resource list.
- The repository contains PowerPoint source files (pptx) for drawing the evolutionary tree diagram, and a static version has been released.
- Contributions to refine the diagram via pull request are welcome.
- If using the resources, it is recommended to cite the original paper.
Applicable scenarios:
- Researchers and practitioners learning about the development history and main models of LLMs.
- Developers choosing a model suitable for their NLP application (LLM or fine-tuned model).
- Users who need to understand model and data license restrictions.
- Individuals learning about LLM-related papers and resources.
Limitations:
- This repository is a resource list and does not provide the models themselves or directly runnable code.
- Usage restriction information is based on model and data licenses; for specifics, refer to the original licenses.
- Some resources may be updated over time; check the latest version.
- The repository content is in English and currently has no Chinese translation.
