This project is a materials repository compiled around work and background technologies related to OpenAI o1. Its content is organized in the form of a paper list, covering directions such as LLM reasoning, self-play reinforcement learning, complex logical reasoning, and scaling laws. The repository is divided by topic, including complex logical reasoning, Reasoning Bootstrapping, Reasoning Scaling Law, Self-play Learning, Step-wise and Process-based Optimization, Applications beyond Math, and Open-source Projects.
In the complex logical reasoning section, it includes papers such as generative language modeling for automated theorem proving, Hypothesis Search, tests of inductive reasoning ability, training verifiers for math word problems, and discussions on Chain-of-thought (CoT) mainly helping mathematical and symbolic reasoning. The reasoning bootstrapping section includes works such as STaR, Quiet-STaR, training chain-of-thought through latent variable inference, chain-of-thought reasoning without prompting, and mutual reasoning improving the problem-solving ability of small models. The reasoning scaling law section involves repeated sampling to scale inference compute, comparisons between test-time compute and model parameter scaling, empirical analysis of compute-optimal reasoning, training models to self-correct through reinforcement learning, and exploration of runtime strategies from Medprompt to o1.
The self-play learning section includes AlphaZero-like work, language model self-programming, LLM self-improvement, self-play fine-tuning, self-play preference optimization, scalable online planning, generative verifiers, Monte Carlo tree self-refinement, and interpretable contrastive Monte Carlo tree search reasoning. The step-wise and process-based optimization section includes process and outcome feedback for solving math problems, fast and slow thinking with tree search, step-wise verification, OVM, LLM critics, self-critique models, automated process supervision, Q*, Math-shepherd, and others. The applications beyond math section includes HuatuoGPT-o1 for complex medical reasoning and o1-Coder for programming. The open-source projects list O1-Journey, OpenReasoner, and HuatuoGPT-o1.
Usage: mainly as an index of research materials, browse the paper titles, author, and institution information in the repository, and access the corresponding papers through links. This repository is a compilation of papers and materials, and does not provide directly runnable tools or models. Applicable scenarios are for researchers and developers to understand the context of reasoning technologies and background literature related to OpenAI o1. Limitations: the materials do not provide specific implementation methods, datasets, installation commands, or usage steps; they also do not provide information such as fees, prices, platforms, or effects. These contents are not yet provided.
