LLMs can be peer pressured, but this work shows how to make them independent thinkers.
This paper studies conformity in multi-agent systems driven by LLMs. It introduces a benchmark and proposes mitigation strategies.
-----
Paper - https://arxiv.org/abs/2501.13381
🤔 Original Problem :
→ LLMs are increasingly used in multi-agent systems.
→ However, their tendency to conform to group opinions, similar to human conformity bias, remains unexplored.
→ This poses risks to their collaborative problem-solving capabilities and ethical implications.
-----
💡:Solution in this Paper :
→ This paper presents BENCHFORM, a conformity-oriented benchmark.
→ BENCHFORM uses reasoning-intensive tasks and five interaction protocols. These protocols explore LLMs’ behavior in short-term and long-term collaborative scenarios.
→ The study also explores two mitigation strategies. These are enhanced personas and a reflection mechanism to encourage independent decision-making.
-----
Key Insights from this Paper :👨🔧
→ All evaluated LLMs exhibit a tendency to conform, impacting their performance in collaborative tasks.
→ Model size correlates positively with independence rates, suggesting larger LLMs are more capable of independent decisions.
→ Individual models show distinct characteristics, with some exhibiting higher credulity or resistance to external guidance.
→ Interaction time and majority size are key factors influencing conformity.
-----
Results : 💯:
→ Conformity rates range from 2.5% to 38.6% across various LLMs and interaction protocols.
→ The Doubt protocol most effectively misleads LLMs, with an average conformity rate of 47.2%.
→ Qwen2-72B exhibits the highest independence rate (57.6%).
→ Enhanced personas and reflection mechanisms increase independence rates up to 13.2% and 40%, respectively, depending on the LLM and protocol.
Share this post