SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI developer platformsopenr

openr

OpenR is an open-source framework for advanced reasoning with large language models, maintained by the Openreasoner team and released under the MIT License. The project provides resources such as papers, tutorials, code, documentation, data, models, issue tracking, and demos, and has released the MATH-APS dataset and the MATH-psa process reward model. OpenR supports MCTS reasoning and provides benchmarks. You first need to download the specified base models (such as Qwen2.5-Math-1.5B-Instruct, Qwen2.5-Math-7B-Instruct, peiyi9979/mistral-7b-sft, peiyi9979/math-shepherd-mistral-7b-prm), and configure the model path, policy model name, value model name, and the number of LM/RM workers by modifying the scripts under reason/llm_service/. After that, you can prepare and run reasoning with different techniques. Before training, you need to modify the dataset path, model path, and PRM path in train/mat/scripts/train_llm.sh. The project welcomes community contributions, not limited to code, but also including answering questions, improving documentation, and more.

Open website

About this tool

OpenR is an open-source framework designed to use large language models for advanced reasoning. The project is maintained by the Openreasoner team and released under the MIT License.

Main purpose: to provide an open-source framework for advanced reasoning with large language models, supporting process reward models, MCTS reasoning, and related training and evaluation.

Main capabilities:

  • Provides the MATH-APS dataset (the project's own dataset) and the MATH-psa process reward model;
  • Supports MCTS reasoning (added on October 24, 2024);
  • Provides benchmarks (Benchmark);
  • Supports RL training and evaluation of search strategies;
  • Project resources include papers, tutorials, code, documentation, data, models, issue tracking, and a demo page (the ModelScope demo page was added on November 29, 2024).

How to use:

  1. Download the required base models: Qwen2.5-Math-1.5B-Instruct, Qwen2.5-Math-7B-Instruct, peiyi9979/mistral-7b-sft, peiyi9979/math-shepherd-mistral-7b-prm. You can refer to the Hugging Face model download tutorial, and make sure all models are saved in the corresponding directories according to the project settings.
  2. Before running inference, modify the variables in the scripts under reason/llm_service/: $MODEL_BASE (model storage directory), $POLICY_MODEL_NAME (policy model name), $VALUE_MODEL_NAME (value model name), $NUM_LM_WORKER (number of language model worker processes to start), $NUM_RM_WORKER (number of reward model worker processes to start).
  3. Prepare and run reasoning with different techniques. For example, to start the LM and RM services for the Math Shepherd model, you can run the corresponding commands; to terminate the service processes, it is recommended to use the corresponding command. Note that the inputs in the script (--LM, --RM) should be consistent with the variables in the worker processes to be started ($POLICY_MODEL_NAME, $VALUE_MODEL_NAME).
  4. Before training, modify $dataset_path, $model_name_or_path, and $prm_name_or_path in train/mat/scripts/train_llm.sh.

Applicable scenarios: research related to mathematical reasoning, training and evaluation of process reward models, MCTS reasoning experiments, RL training and search strategy evaluation, etc. The project also lists related references, covering directions such as inference-time computation, from outcome supervision to process supervision, and data acquisition.

Limitations: the materials do not provide specific performance metrics, pricing information, platform compatibility details, or complete installation commands. Before use, you need to download and configure the base models yourself. The project welcomes community contributions, including code, answering questions, helping others, improving documentation, and sharing the project.

Related tools

Browse category