MoA

Mixture of Agents (MoA) is an open-source project aimed at improving the quality of final answers by combining the collective strengths of multiple large language models (LLMs). The method uses a layered architecture, where each layer contains several LLM agents, and through multiple rounds of aggregation and refinement, generates responses superior to those of a single model. According to official materials, using only open-source models, MoA achieved a score of 65.1% on AlpacaEval 2.0, while GPT-4 Omni scored 57.5%, achieving an absolute improvement of 7.6%.

Purpose: MoA is mainly used to improve the quality of LLM responses on complex tasks, and is suitable for scenarios requiring high-quality text generation, multi-turn dialogue, and model ensemble evaluation. It can aggregate the outputs of multiple reference models to generate a more accurate and comprehensive final answer.

Main capabilities:

• Multi-layer MoA architecture: supports layered processing with 2 or more layers, each layer containing multiple LLM agents, refining answers layer by layer.
• Interactive CLI demo: provides a multi-turn chatbot demo, where the final response is aggregated from multiple reference models, supporting multi-turn interaction.
• Evaluation scripts: includes evaluation code for AlpacaEval 2, MT-Bench, and FLASK, facilitating reproduction of the paper's results.
• Achieves leading rankings on both AlpacaEval 2.0 and MT-Bench. In FLASK's fine-grained evaluation, MoA significantly outperforms the original Qwen1.5-110B-Chat in dimensions such as harmlessness, robustness, correctness, efficiency, factuality, common sense, insightfulness, and completeness, and outperforms GPT-4 Omni in correctness, factuality, insightfulness, completeness, and metacognition.

Usage: Quick start requires installing the Together Python library and setting the API key:

1. Install: pip install together
2. Export API key: export TOGETHER_API_KEY=<your key>
3. Run example: python moa.py (a simple example containing 2 layers and 4 LLMs)

A multi-layer MoA example can be run with advanced-moa.py, supporting 3 or more layers.

Interactive CLI demo steps:

1. Export API key: export TOGETHER_API_KEY={your_key}
2. Install dependencies: pip install -r requirements.txt
3. Run script: python bot.py Enter instructions at the ">>>" prompt, and the system will process the input using predefined reference models and generate a response based on the aggregated output, supporting multi-turn dialogue.

Optional configuration parameters include: --aggregator: the main model used for final response generation. --reference_models: the list of models used as references. --temperature: controls the randomness of response generation. --max_tokens: the maximum number of tokens in the response. --rounds: the number of rounds for processing input (rounds = number of MoA layers - 1). --num_proc: the number of processes run in parallel to speed up execution. --multi_turn: a boolean value that toggles multi-turn interaction capability.

Evaluation runs: the project provides scripts for running AlpacaEval 2, MT-Bench, and FLASK evaluations, including the necessary modifications.

Applicable scenarios:

• Research and development requiring high-quality text generation.
• Multi-model ensemble and comparative evaluation.
• Building multi-turn dialogue systems.
• Reproducing the benchmark results in the paper.

Limitations:

• Depends on the Together API and API key.
• No detailed information is provided regarding charges, pricing, or specific performance guarantees; not yet provided.
• Official materials do not explain other supported platforms or local deployment methods.
• The project is under the Apache 2.0 license.