L2MAC (Large Language Model Automatic Computer) is a general-purpose stored-program automatic computer framework based on LLMs, adopting the von Neumann architecture, for LLM multi-agent systems. It is designed to solve complex tasks by generating large amounts of consistent output, unconstrained by the LLM's fixed context window. The framework was fully open-sourced on April 23, 2024, and was presented at ICLR 2024.
Purpose: L2MAC excels at solving large, complex tasks, such as generating large codebases or writing entire books, which are typically constrained by the LLM's fixed context window. It can generate extensive and large amounts of output from a single-line input prompt, such as generating an entire complex codebase.
Key capabilities: L2MAC internally uses a prompt-program, i.e., a series of prompts, each providing an instruction step to be executed. Unless explicitly given, the prompt-program is self-generated (bootstrapped) and executed. Each instruction step is loaded into a new LLM agent for execution, whose context is managed by a control unit, and tools are provided to read and write persistent memory (file storage), which contains final and intermediate outputs. This enables it to automatically execute general-purpose prompt-programs to solve complex tasks requiring large amounts of coherent output, with output unconstrained by the LLM's underlying context window.
Usage: Ensure the system has Python 3.7+ installed. You can create an environment using conda: conda create -n l2mac python=3.9 && conda activate l2mac. For detailed installation instructions, please refer to the installation documentation. You can initialize the L2MAC configuration by running a command, or manually create the ~/.L2MAC/config.yaml file. Configure ~/.l2mac/config.yaml according to the examples and documentation. After installation, you can use the L2MAC CLI or use it as a library.
Applicable scenarios: L2MAC is suitable for generating large, complex codebases (such as entire applications), writing entire books, and more. Examples include: a playable Snake game (using pygame), a large, complex URL-shortening web application, an Italian pasta recipe book (26 pages), and a playable blackjack CLI game.
Limitations: No explicit limitation information is currently provided. However, according to the description, L2MAC depends on LLMs and may be limited by the capabilities of the LLM itself.
Performance: It achieves 90.2% Pass@1 on the HumanEval benchmark and ranks third on the global coding industry standard leaderboard. Compared with AutoGPT and GPT-4, L2MAC performs better in retaining user-specified functionalrequirements, file access, and error correction.
Contact information: If you have questions or feedback, you can contact sih31@cam.ac.uk, or create an issue in the GitHub repository, and you will receive a reply within 2-3 business days. Follow Twitter @samianholt for the latest research updates.
