Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System

LLM based multi-agent systems (MAS) for collaborative problem-solving.

https://arxiv.org/abs/2410.08115

Original Problem 🔍:

LLM-based multi-agent systems (MAS) face challenges in communication efficiency, scalability, and optimization methods.

-----

Solution in this Paper 🧠:

OPTIMA framework:

• Iterative generate, rank, select, and train paradigm

• Reward function balancing task performance, token efficiency, and readability

• Explores SFT, DPO, and hybrid approaches

• Integrates MCTS-inspired techniques for DPO data generation

-----

Key Insights from this Paper 💡:

• OPTIMA significantly improves both communication efficiency and task effectiveness

• Demonstrates potential for enhancing inference-time scaling laws

• Shows ability to transfer knowledge to out-of-distribution tasks

• Highlights importance of efficient communication in MAS and LLM systems

-----

Results 📊:

• Up to 2.8x performance gain with <10% tokens on information exchange tasks

• Consistent outperformance over single-agent baselines and vanilla MAS

• Improved inference-time scaling laws

• Effective knowledge transfer to out-of-distribution tasks