What is MaskSearch
MaskSearch is a novel general-purpose pre-training framework launched by Alibaba Tongyi Lab, enhancing the agentic search capabilities of large language models (LLMs). Through the Retrieval-Augmented Mask Prediction (RAMP) task, the model masks key information in the input text and uses external knowledge bases to invoke search tools to predict the masked segments, covering key information such as named entities, dates, numbers, and ontological knowledge, increasing task difficulty and prompting the model to process information in a more fine-grained manner. When generating supervised fine-tuning (SFT) data, a multi-agent system is used, including roles such as planner, rewriter, and observer, to collaboratively generate chain-of-thought data and improve data quality. The training method combines SFT and reinforcement learning (RL), uses the Dynamic Sampling Policy Optimization (DAPO) algorithm to build a hybrid reward system, and adopts curriculum learning, grading sample difficulty by the number of masks, allowing the model to learn progressively.
Main Features of MaskSearch
- Improved question-answering performance: Significantly enhances LLM performance in open-domain multi-hop question-answering scenarios, especially on in-domain and out-of-domain downstream tasks, improving the model's ability to understand and answer complex questions.
- Adaptability to multiple tasks: Through the RAMP task and chain-of-thought data generated by multiple agents, the model can better adapt to various question-answering tasks and improve performance in different scenarios.
- Compatibility with multiple training methods: Compatible with both SFT and RL training methods, allowing the selection of appropriate training strategies according to different task requirements.
- Dataset expansion: By constructing a large-scale pre-training dataset (such as 10 million samples), the model's training effectiveness and scalability are improved.
Technical Principles of MaskSearch
- Retrieval-Augmented Mask Prediction (RAMP) task: The core of MaskSearch is the RAMP task, inspired by BERT's masking mechanism, which has the model mask key information in the input text sequence and then actively use external knowledge bases to invoke search tools to predict these masked text segments. In addition to common named entities, dates, and numbers, it also considers key information such as ontological knowledge, specific terms, and numerical values, increasing task difficulty and prompting the model to process information more meticulously during retrieval and reasoning.
- Multi-agent collaborative generation of chain-of-thought (CoT) data: To generate chain-of-thought data for supervised fine-tuning (SFT), MaskSearch uses a multi-agent system, including roles such as planner, rewriter, and observer, to collaboratively carry out chain-of-thought generation tasks, with an LLM finally responsible for answer judgment and retaining only the chain of thought with the correct answer. To quickly expand the dataset while maintaining high quality, a teacher model trained on existing data is used to directly generate reasoning trajectories, and the teacher model is iteratively improved step by step, gradually improving data quality.
- Reinforcement learning (RL): In the reinforcement learning part, MaskSearch adopts the Dynamic Sampling Policy Optimization (DAPO) algorithm to build a hybrid reward system, including format rewards and answer rewards. Format rewards check whether the model output conforms to the specified format, while answer rewards evaluate the consistency between the generated answer and the standard answer. Finally, a model-based reward function is chosen, using the Qwen2.5-72B-Instruct model as the judge to score the consistency between the generated answer and the standard answer.
- Curriculum learning: MaskSearch introduces a curriculum learning strategy, grading the difficulty of training samples according to the number of masks, allowing the model to learn from easy to difficult in sequence and gradually improve its capabilities.
MaskSearch Project Address
- Github repository：https://github.com/Alibaba-NLP/MaskSearch
- arXiv technical paper：https://arxiv.org/pdf/2505.20285
Application scenarios of MaskSearch
- Intelligent customer service: In the field of intelligent customer service, MaskSearch can help customer service systems understand user questions more accurately, quickly retrieve relevant answers and information, and improve customer service efficiency and user satisfaction.
- Education: In the field of education, MaskSearch can be used to build intelligent tutoring systems to help students better understand and solve complex academic problems. Based on students' questions, it can retrieve relevant knowledge points and solutions, providing personalized learning support.
- Enterprise-level search systems: Enterprise-level search systems need to handle large amounts of internal data and complex query requirements. MaskSearch can enhance the retrieval capabilities of enterprise search systems, more accurately understand users' query intent, quickly retrieve relevant information from massive data, and improve enterprise decision-making efficiency.
- Debugging and optimization of machine learning models: MaskSearch can also be used for debugging and optimizing machine learning models. For example, when debugging an image classification model, MaskSearch can help users query an image database through mask attributes, identify spurious correlations learned by the model, and explore the differences between model saliency and human attention.