SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI agentsAI-Scientist

AI-Scientist

AI-Scientist is an open-source system launched by SakanaAI, with the goal of achieving fully automated, open-ended scientific discovery. It enables foundation models such as large language models to conduct research independently: generating ideas, writing code, executing experiments, writing papers, and using LLMs for automated peer review. The project provides three templates, NanoGPT, 2D Diffusion, and Grokking, covering the corresponding research areas. The code is designed to run under Linux, NVIDIA GPU, CUDA, and PyTorch environments, and supports multiple models and API keys. Note: this code will execute code written by LLMs, which poses risks such as using dangerous packages, network access, and launching processes; you need to weigh these risks yourself and use containerization and restrict network access. In the paper, the cost per paper is usually less than $15 (using Claude Sonnet 3.5), and DeepSeek Coder V2 is recommended to reduce costs.

Open website

About this tool

Purpose

AI-Scientist is a system developed by SakanaAI for fully automated open-ended scientific discovery. Its goal is to enable foundation models such as large language models to conduct research independently, rather than merely serving as auxiliary tools. The system is intended to cover the complete scientific research process from idea generation to paper generation and review, reducing human supervision.

Main Capabilities

  • Automatically generate research ideas and execute experiments.
  • Write papers and support LLM-generated paper review.
  • Provide three templates, each targeting a different research area: NanoGPT (autoregressive next-token prediction tasks based on Transformers), 2D Diffusion (improving the performance of diffusion generative models on low-dimensional datasets), and Grokking (generalization and learning speed issues in deep neural networks).
  • Support multiple models, including open-weight and API-only models, and it is recommended to use frontier models more capable than the original GPT-4.
  • Support configuring API keys through environment variables, including OPENAI_API_KEY, ANTHROPIC_API_KEY, DEEPSEEK_API_KEY, OPENROUTER_API_KEY, etc., and support Google Gemini models.
  • Optionally use a Semantic Scholar API Key (S2_API_KEY) to improve throughput, or use the OpenAlex API as an alternative (no API key required, experimental).

Usage

  1. Environment requirements: the code is designed to run under Linux, NVIDIA GPU, CUDA, and PyTorch. For other GPU architectures, refer to the PyTorch guide; the current templates may take an infeasible amount of time on pure CPU machines; other operating systems may require extensive adjustments.
  2. Installation: texlive-full needs to be installed, the process may take a long time, and you may need to hold down Enter during installation.
  3. Set up the template: complete the preparation steps for the selected template.
    • NanoGPT template: prepare the data, run python data/enwik8/prepare.py、python data/shakespeare_char/prepare.py、python data/text8/prepare.py; then create a baseline run (under templates/nanoGPT, execute python experiment.py --out_dir run_0 and python plot.py).
    • 2D Diffusion template: install dependencies, clone NPEET and install it, install scikit-learn; then create a baseline run (under templates/2d_diffusion, execute python experiment.py --out_dir run_0 and python plot.py).
    • Grokking template: install dependencies pip install einops; then create a baseline run (under templates/grokking, execute python experiment.py --out_dir run_0 and python plot.py).
  4. Run the experiment: after completing setup, run the main experiment script; if there are multiple GPUs, you can use --parallel Option parallel processing ideas.
  5. Custom templates: Create according to the existing template structure, including experiment.py, plot.py, prompt.json, seed_ideas.json, latex/template.tex, keeping the base file names and output JSON format consistent, and update the citation style and basic figures in template.tex.
  6. Community templates: Community contributions of new templates are welcome. A list of some community templates and corresponding PR links is provided, such as Infectious Disease Modeling, Image Classification with MobileNetV3, Sketch RNN, AI in Quantum Chemistry, Earthquake Prediction, Tensorial Radiance Fields, Large Language Model Steering / Probes, etc.

Applicable Scenarios

  • Automated scientific research exploration, especially for research ideas that can be expressed in code.
  • Automatically generate ideas and conduct experiments in areas such as NanoGPT, 2D Diffusion, and Grokking.
  • Research workflows that require automatically generated paper drafts and reviews.
  • Researchers or developers exploring AI-driven scientific discovery.

Limitations and Notes

  • This codebase executes code written by an LLM, which carries risks, including the use of potentially dangerous software packages, network access, and the possible launching of processes. Use it at your own discretion, and be sure to containerize and appropriately restrict network access.
  • In the current iteration, the system is limited to ideas that can be expressed with available code.
  • The success rate of paper generation and the quality of reviews depend on the template, base model, and idea complexity; the highest success rate is seen with Claude Sonnet 3.5, and reviews are best with GPT-4o, while other models have issues with positivity bias or failure to conform to the required output.
  • The cost per idea is usually less than 15 USD (using Claude Sonnet 3.5); DeepSeek Coder V2 is recommended to significantly reduce costs.
  • You need to run the baseline run (run_0) yourself, because hardware differences affect runtime comparisons.
  • If you cannot access the Semantic Scholar API, the literature search and citation stages may be skipped.
  • The project uses The AI Scientist Source Code License (a derivative of the Responsible AI License). When using it, you must clearly disclose the use of AI. It is recommended to note in the paper abstract or methods section: "This manuscript was autonomously generated using The AI Scientist."
  • A community-contributed Docker image (experimental/Dockerfile) is provided and can be used for containerization.
  • By default, the OPENAI_API_KEY environment variable, etc., is used, and you need to provide the key for the model used in the run.

Related tools

Browse category