Search-R1 is a reinforcement learning training framework whose goal is to train "reasoning-search interleaved" large language models—that is, to let the model learn to coordinate tool calls (for example, calling a search engine) during reasoning. The project is built on veRL and extends the ideas of DeepSeek-R1(-Zero) by introducing interleaved search engine access, providing a fully open-source RL training pipeline, and can serve as a substitute and open solution for OpenAI DeepResearch, for research and development on tool-augmented LLM reasoning.
Main capabilities and support scope:
- Supports different RL methods, such as PPO, GRPO, and reinforce.
- Supports different LLMs, such as llama3, Qwen2.5, etc.
- Supports different search engines: local sparse retrievers (such as BM25), local dense retrievers (GPU tiled index and CPU ANN index), as well as online search engines (such as Google, Bing, Brave Search API, etc.).
- Supports off-the-shelf neural rerankers.
- Supports multi-node training for 30B+ LLMs.
- Has been integrated into the latest version of veRL and is supported by SkyRL.
Usage: The project provides guidance such as "use your own dataset" and "use your own search engine". The core idea is to start the local or remote search engine server separately from the main RL training pipeline, and the LLM uses the search API (for example, http://127.0.0.1:8000/retrieve）调用搜索引擎。若使用本地检索器作为搜索引擎，需单独安装环境。
The process of training a reasoning + search LLM on the NQ dataset with e5 as the retriever and Wikipedia as the corpus includes:
- Download the index and corpus.
- Process the NQ dataset.
- Start the local retrieval server.
- Run PPO reinforcement learning training with Llama-3.2-3b-base.
An example inference process is: first start the local retrieval server, then run inference; the question on line 7 can be modified.
Dataset format: each question-answer sample should be a dictionary containing the required content. For a specific data processing example, refer to scripts/data_process/nq_search.py. The corpus is recommended to be a jsonl file, with each line being a dictionary containing an "id" key and a "contents" key, corresponding to a paragraph; "id" corresponds to the paragraph id, and "contents" corresponds to the paragraph content ('"' + title + '"\n' + text). For an example, see example/corpus.jsonl.
Index: if you want to use a local retriever as the search engine, you can build an index for your own corpus, and you can modify retriever_name and retriever_model to an off-the-shelf retriever of interest.
Preliminary results: the base model llama3.2-3b-base can learn to call the search engine and achieve performance improvements; the Qwen2.5-7b-base base model can learn through RL to perform multiple rounds of search engine calls and reasoning.
Project status: Search-R1 open-sourced the codebase and gave preliminary results in February 2025; released the first paper in March 2025 (v0.1, v0.2 logs); in April 2025 supported multi-node training for 30B+ LLMs and supported multiple search engines; in May 2025 released the second paper and v0.3 logs; in June 2025 was integrated into the latest version of veRL; in July 2025 received SkyRL support; in October 2025 was included in Thinking Machines Lab's first product, Tinker.
The concept of Search-R1 is inspired by Deepseek-R1 and TinyZero, and the implementation is built on veRL and RAGEN. Papers, models and data, Twitter thread, complete experiment logs (prelim, v0.1, v0.2, v0.3) and other links are given in the project materials. In addition, the materials list multiple works inspired by or built upon it, such as DeepResearcher, Multimodal-Search-R1, OTC, ZeroSearch, IKEA, Scent of Knowledge, AutoRefine, O^2-Searcher, MaskSearch, VRAG-RL, R1-Code-Interpreter, R-Search, StepSearch, SimpleTIR, Router-R1, SkyRL, ASearcher, ParallelSearch, AutoTIR, verl-tool, Tree-GRPO, EviNote-RAG, GlobalRAG, etc. The specific installation commands and complete tutorial must be found in the project repository; the materials do not provide the full command text.
