What Is RAG-FiT
RAG-FiT (formerly known as RAG Foundry) is an open-source framework launched by Intel Labs for fine-tuning to enhance the performance of large language models (LLMs) in retrieval-augmented generation (RAG) tasks. RAG-FiT is based on a modular design and supports four major functional modules: data creation, training, inference, and evaluation. RAG-FiT helps users quickly create datasets suitable for RAG tasks, optimize models using parameter-efficient fine-tuning (PEFT) techniques, and measure model performance using a variety of RAG-specific evaluation metrics. The RAG-FiT framework offers a high degree of flexibility and extensibility, supporting comprehensive RAG use cases from data selection and filtering to retrieval and prompt generation, and is suitable for a variety of application scenarios such as question-answering systems and text generation.
Main Features of RAG-FiT
- Data Creation and Processing：
- Data Loading: Supports loading datasets from Hugging Face Hub or local sources.
- Data Preprocessing: Includes data filtering, normalization, aggregation, information retrieval, templated prompt generation, and more.
- Data Saving: Processed data is saved in a consistent format for subsequent training and inference.
- Flexible Processing Pipeline: Supports global operations (such as data aggregation and filtering) and local operations (such as retrieval and text processing for individual samples).
- Training：
- Parameter-Efficient Fine-Tuning (PEFT): Uses techniques such as LoRA for efficient model fine-tuning.
- Training Configuration: Supports custom training parameters such as learning rate, optimizer, and batch size.
- Model Pushing: Trained models can be pushed to Hugging Face Hub.
- Inference：
- Generate Predictions: Generates prediction results on the processed dataset.
- Batch Inference: Supports batch processing of multiple input data to improve efficiency.
- Evaluation：
- Multi-Dimensional Evaluation: Supports a variety of evaluation metrics such as EM, F1, ROUGE, BERTScore, and more.
- Custom Evaluation: Users can easily implement custom evaluation metrics.
- Global and Local Evaluation: Supports local evaluation for each sample and global evaluation for the entire dataset.
Technical Principles of RAG-FiT
- Retrieval Augmentation：
- Retrieval Mechanism: Retrieves context information relevant to the input question from external knowledge bases using retrieval tools. Tools include vector-based retrieval systems (such as Haystack, Qdrant) and other retrieval frameworks.
- Context Injection: Injects the retrieved context information into the input of LLMs, helping the model better understand the question background and thus generate more accurate and better-grounded answers.
- Modular Design：
- Data Processing Module: Responsible for data loading, preprocessing, and saving. Supports multiple data sources and flexible processing pipelines.
- Training Module: Uses PEFT techniques to fine-tune models and supports efficient training methods such as LoRA. Trained models can be saved or pushed to Hugging Face Hub.
- Inference Module: Generate prediction results on the processed dataset, supporting batch inference.
- Evaluation module: Provides a variety of evaluation metrics and supports multi-dimensional evaluation of generated results.
- Configuration-driven workflow：
- Hydra configuration tool: Uses the Hydra configuration tool to implement hierarchical configuration and supports overriding configuration values via the command line, facilitating remote job execution.
- Configuration files: Each module has a default configuration file, and users can customize the workflow based on configuration files or command-line parameters.
- Experimentation and evaluation：
- Experimental environment: Provides an end-to-end experimental environment that supports rapid prototyping and experimentation with various RAG techniques.
- Multi-dimensional evaluation: The evaluation module evaluates the accuracy of generated results, and can also evaluate the relevance of retrieval results and the faithfulness and relevancy of generated content.
Project address of RAG-FiT
- Project official website：https://intellabs.github.io/RAG-FiT/
- GitHub repository：https://github.com/IntelLabs/RAG-FiT
- arXiv technical paper：https://arxiv.org/pdf/2408.02545
Application scenarios of RAG-FiT
- Question answering systems: Enhances language models based on retrieval from external knowledge bases to provide users with more accurate and relevant answers, suitable for professional fields such as medicine and law.
- Text generation: Generates high-quality text in combination with the latest background information, such as news reports and creative writing, improving the timeliness and accuracy of the content.
- Knowledge graph enhancement: Retrieves entities and relations from knowledge graphs and generates text consistent with the graph, improving the accuracy and interpretability of knowledge representation.
- Multilingual generation: Retrieves knowledge bases across languages and generates multilingual text, meeting content generation needs in multilingual environments.
- Document summarization: Retrieves key information from documents to generate summaries, improving the accuracy and information coverage of summaries, suitable for fields such as scientific research and business.