g1 is an experimental prototype designed to improve the reasoning capabilities of large language models (LLMs) through prompting strategies, generating o1-like reasoning chains. It is based on the Groq platform and uses the open-source model Llama-3.1 70b, allowing the model to "think" step by step and solve logic problems that typically stump leading models. Unlike OpenAI's o1, g1 displays all reasoning tokens and uses an open-source model; it is not a complete reproduction of or comparison with o1, but rather is intended to inspire the open-source community to develop new reasoning strategies.
Main capabilities: g1 generates reasoning chains through dynamic Chain of Thought, with each step having a title and being visible to the user. At each step, the model can choose to continue reasoning or provide a final answer. The system prompt includes various techniques, such as asking the model to "explore alternative answers," "derive the answer using at least 3 methods," and "consider its own limitations," thereby improving reasoning ability. On the strawberry problem ("How many R's are in strawberry?"), g1 has an accuracy of about 70% (n=10), while unprompted Llama-3.1-70b is 0% and ChatGPT-4o is 30%. Initial tests show that g1 can achieve 60-80% accuracy on simple logic problems that typically stump LLMs, but the accuracy has not yet been formally evaluated.
How to use: Users can use g1 through the Streamlit UI or Gradio UI. For specific installation and run commands, please refer to the instructions in the GitHub repository. The repository provides a detailed breakdown of the prompting strategy, including role setting, step-by-step reasoning instructions, JSON format requirements (containing the 'title', 'content', and 'next_action' keys), and all-caps best-practice prompts.
Use cases: g1 is suitable for tasks requiring logical reasoning, and is especially suitable for solving simple logic problems that typically stump LLMs, such as the strawberry problem and comparisons of magnitude. It can help researchers and developers explore the improvement of reasoning ability through prompt engineering.
Limitations: g1 is an early experimental prototype and is not perfect; its accuracy has not been formally evaluated. It relies on prompting strategies rather than training, so its performance on different problems may be unstable. In addition, g1 uses the Groq platform and the Llama-3.1 70b model, and may be limited by platform availability and model performance.
