What Is AgentPrune
AgentPrune is a communication optimization framework for large language model (LLM)-driven multi-agent systems, jointly proposed by Tongji University, The Chinese University of Hong Kong, and other institutions. It uses "pruning" techniques to remove redundant or harmful communication content, reducing communication costs and improving system performance. AgentPrune models multi-agent systems as spatiotemporal graphs, optimizes communication connections based on low-rank sparse graph masks, and generates efficient communication topologies through one-shot pruning. AgentPrune performs excellently in multiple benchmarks, achieving comparable performance at only 5.6% of the cost of traditional methods. It can seamlessly integrate into existing multi-agent frameworks such as AutoGen and GPTSwarm, achieving token reductions of 28.1% to 72.8%.
Main Features of AgentPrune
- Communication Redundancy Identification and Pruning: AgentPrune is the first to identify and define the problem of communication redundancy in LLM multi-agent systems. Through one-shot pruning, it removes redundant and harmful communication content.
- Spatiotemporal Graph Modeling and Optimization: It models multi-agent systems as spatiotemporal graphs, including spatial edges (communication within the same round of dialogue) and temporal edges (communication across rounds of dialogue), and optimizes them through parameterized graph masks.
- Application of Low-Rank Sparse Graph Masks: Based on low-rank sparse graph masks, it promotes a sparser communication structure, reduces redundancy, noise, and malicious messages, and enhances robustness against network attacks.
- Cost and Performance Optimization: In multiple benchmarks, AgentPrune achieves results comparable to existing state-of-the-art topologies at a lower cost (only $5.6 compared to $43.7 for other systems), and can seamlessly integrate into existing multi-agent frameworks, achieving token reductions of 28.1% to 72.8%.
- Adversarial Attack Defense: AgentPrune can successfully defend against two types of agent adversarial attacks, bringing performance improvements of 3.5% to 10.8%.
Technical Principles of AgentPrune
- Spatiotemporal Graph Modeling: AgentPrune models the communication structure of a multi-agent system as a spatiotemporal graph, where nodes represent agents and edges represent communication connections, divided into spatial edges (communication within the same round of dialogue) and temporal edges (communication across rounds of dialogue).
- Parameterized Graph Masks: AgentPrune optimizes communication connections through parameterized graph masks. The goal of the graph mask is to reflect the importance of communication connections through distribution approximation and low-rank sparsity. Distribution approximation maximizes the utility of the system through policy gradient methods while minimizing communication redundancy; low-rank sparsity uses low-rank constraints to promote a sparser communication structure, removing redundant, noisy, and even malicious messages.
- One-Shot Pruning: In the early stage of training, AgentPrune performs a limited number of optimizations on the graph mask, and then removes unimportant communication connections through one-shot pruning. Specifically, AgentPrune selects a certain proportion of the most important connections to retain based on the magnitude of the graph mask, generating a sparse communication graph.
- Optimized Communication Graph: In the subsequent communication process, the multi-agent system strictly follows this optimized communication graph for message passing, reducing communication costs while maintaining efficient performance.
AgentPrune Project Address
- GitHub Repository：https://github.com/yanweiyue/AgentPrune
- arXiv Technical Paper：https://arxiv.org/pdf/2410.02506
Application Scenarios of AgentPrune
- Multi-Agent System Optimization: AgentPrune can be seamlessly integrated into existing multi-agent frameworks such as AutoGen and GPTSwarm, significantly reducing communication costs while maintaining or improving system performance.
- Cost-effective communication topology: In multi-agent systems, AgentPrune uses one-shot pruning techniques to generate sparse communication topologies, significantly reducing token consumption.
- Complex task collaboration: AgentPrune is suitable for complex tasks that require collaboration among multiple agents, such as mathematical reasoning, code generation, and commonsense question answering. By optimizing the communication structure, AgentPrune can improve task completion efficiency and reduce economic costs.
- Industrial and enterprise-level applications: In industrial automation and enterprise-level applications, AgentPrune can optimize communication between agents, reduce resource waste, and improve the overall efficiency of the system.