SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI developer platformsstorm

storm

STORM is an LLM-based knowledge organization system developed by the Stanford OVAL team that can conduct web research on a topic and generate long-form reports with citations. The system consists of two stages, "Research and Outline" and "Writing," and enhances the depth and breadth of questions through multi-perspective questioning and simulated dialogue; its collaborative version, Co-STORM, introduces a human-machine collaborative discussion protocol, multiple LLM agents (experts, moderators), and a dynamic mind map. Users can install it via pip install knowledge-storm, and it supports various language models and embedding models from litellm, as well as multiple retrieval sources such as You.com, Bing, Serper, Brave, DuckDuckGo, and Tavily, and also supports VectorRM based on self-built documents. It is suitable for knowledge exploration and material organization in the early stages of writing, but the official documentation states that its output cannot yet be used directly as a publishable article.

Open website

About this tool

STORM (Synthesis of Topic Outlines through Retrieval and Multi-perspective Question Asking) is an LLM-driven knowledge organization system that can research a topic and generate full-length reports with citations. The project was developed by the Stanford OVAL team and is currently a research preview, with more than 70,000 people having tried the online preview.

Purpose and Positioning STORM is aimed at knowledge exploration and early-stage writing preparation. The official documentation clearly states that the system currently cannot produce publishable articles, which usually require extensive editing, but experienced Wikipedia editors consider it helpful in the early stages of writing.

How It Works STORM breaks down the generation of long-form cited articles into two steps:

  1. Pre-writing stage: The system conducts internet-based research, collects references, and generates an outline.
  2. Writing stage: The system uses the outline and references to generate a complete article with citations. The official view is that the core of the automated research process lies in automatically asking good questions, and directly prompting a language model to ask questions does not work well. To improve the depth and breadth of questions, STORM adopts two strategies:
  • Perspective-Guided Question Asking: Given an input topic, STORM discovers different perspectives by researching existing articles on similar topics and uses these perspectives to guide the questioning process.
  • Simulated Conversation: STORM simulates a conversation between a Wikipedia writer and a domain expert based on internet sources, enabling the language model to update its understanding of the topic and ask follow-up questions.

Co-STORM Collaboration Mode Co-STORM proposes a collaborative discourse protocol that, through turn management strategies, supports smooth collaboration among the following three parties:

  • Co-STORM LLM expert: Generates answers based on external knowledge sources and/or asks follow-up questions based on the discourse history.
  • Moderator: Inspired by information discovered by the retriever but not directly used in previous turns, generates thought-provoking questions; question generation can also be based on sources.
  • Human user: Can actively choose to (1) observe the discourse to gain a deeper understanding of the topic, or (2) actively participate in the conversation by injecting discourse to guide the focus of discussion. Co-STORM also maintains a dynamically updated mind map (nd map), organizing the collected information into a hierarchical conceptual structure, striving to establish a shared conceptual space between human users and the system. It is officially claimed that this mind map has been proven to help reduce cognitive load when discourse becomes lengthy and in-depth.

Technical Implementation and Model Support STORM and Co-STORM are both implemented in a highly modular manner using dspy. The current package supports:

  • Language model components: all language models supported by litellm.
  • Embedding model components: all embedding models supported by litellm.
  • Retrieval module components: YouRM, BingSearch, VectorRM, SerperRM, BraveRM, SearXNG, DuckDuckGoSearchRM, TavilySearchRM, GoogleSearch, and AzureAISearch.

Installation and Usage It can be installed via pip: pip install knowledge-storm; the source code can also be installed to directly modify the behavior of the STORM engine: git clone https://github.com/stanford-oval/storm.git cd storm Source environment installation: conda create -n storm python=3.11 conda activate storm pip install -r requirements.txt Both STORM and Co-STORM operate at the information organization layer, and the information retrieval module and language model module must be set up first before the respective Runner classes can be created. The STORM knowledge organization engine is defined as the Python STORMWikiRunner class, which can be invoked via the run method, and its main parameters include:

  • do_research: when True, simulate conversations from different perspectives to collect topic information; otherwise, load existing results.
  • do_generate_outline: when True, generate an outline for the topic; otherwise, load existing results.
  • do_generate_article: when True, generate an article based on the outline and the collected information; otherwise, load existing results.
  • do_polish_article: when True, polish the article by adding a summary section and (optionally) removing duplicate content; otherwise, load existing results. The Co-STORM knowledge organization engine is defined as the Python CoStormRunner class, which can be invoked via the warmstart() and step(...) methods.

Examples and Configuration Official scripts are provided in the examples folder to quickly run STORM and Co-STORM with different configurations, and it is recommended to use secrets.toml to set API keys. The gpt series models with the default configuration can be run, and other language models or one's own corpus can also be used; the Co-STORM example requires adding BING_SEARCH_API_KEY and ENCODER_API_TYPE to secrets.toml.

Pipeline Customization If the source code is installed, STORM can be customized based on one's own use case. The STORM engine consists of 4 modules:

  • Knowledge organization module: collects broad information about a given topic.
  • Outline generation module: organizes the collected information by generating a hierarchical outline for the organized knowledge.
  • Article generation module: fills the generated outline with the collected information.
  • Article polishing module: improves and enhances the article to improve its presentation. The interfaces of each module are defined in knowledge_storm/interface.py, with implementations located in knowledge_storm/storm_wiki/modules/*, and can be customized as needed (for example, generating subsections in bullet-point format instead of full paragraphs). Co-STORM is also customizable: its LLM agent interfaces are defined in knowledge_storm/interface.py, with implementations located in knowledge_storm/collaborative_storm/modules/co_storm_agents.py; the core of the collaborative discourse protocol is turn policy management, with an example implementation being the DiscourseManager in knowledge_storm/collaborative_storm/engine.py.

Datasets The project has released the following datasets to facilitate research on automated knowledge curation and complex information retrieval:

  • FreshWiki dataset: contains 100 high-quality Wikipedia articles, focusing on the most-edited pages between February 2022 and September 2023. It can be downloaded directly from huggingface, and the source code of the data construction pipeline has been archived for future reproduction.
  • WildSeek dataset: data collected from web research previews, downsampled to ensure topic diversity and data quality. Each data point contains a topic and the user's goal for deep searching on that topic.

Paper Reproduction and Roadmap The STORM paper experiments can be switched to the NAACL-2024-code-backup branch; the Co-STORM paper experiments correspond to the EMNLP-2024-code-backup branch (officially marked as a placeholder and will be updated). Directions the team is advancing include: Human-in-the-Loop functionality, supporting users' participation in the knowledge curation process; information abstraction, developing abstractions for curated information to support presentation formats beyond Wikipedia-style reports.

License and Acknowledgments The project thanks Wikipedia for its open content. The FreshWiki dataset is derived from Wikipedia and is licensed under CC BY-SA.

Limitations The official statement notes that the system cannot yet generate finished articles ready for publication and usually requires extensive editing; it is positioned as an auxiliary tool for pre-writing and research exploration.

Related tools

Browse category