This Is Your Doge, If It Please You: Exploring Deception and Robustness in Mixture of LLMs
This addresses safety and reliability concerns for AI systems using collaborative LLMs, highlighting critical vulnerabilities in a state-of-the-art approach.
The study investigates the vulnerability of Mixture of LLMs (MoA) architectures to deceptive agents, showing that a single deceptive agent can reduce performance on AlpacaEval 2.0 from 49.2% to 37.9% Win Rate and cause a 48.5% accuracy drop on QuALITY, and proposes defense mechanisms to mitigate these effects.
Mixture of large language model (LLMs) Agents (MoA) architectures achieve state-of-the-art performance on prominent benchmarks like AlpacaEval 2.0 by leveraging the collaboration of multiple LLMs at inference time. Despite these successes, an evaluation of the safety and reliability of MoA is missing. We present the first comprehensive study of MoA's robustness against deceptive LLM agents that deliberately provide misleading responses. We examine factors like the propagation of deceptive information, model size, and information availability, and uncover critical vulnerabilities. On AlpacaEval 2.0, the popular LLaMA 3.1-70B model achieves a length-controlled Win Rate (LC WR) of 49.2% when coupled with 3-layer MoA (6 LLM agents). However, we demonstrate that introducing only a $\textit{single}$ carefully-instructed deceptive agent into the MoA can reduce performance to 37.9%, effectively nullifying all MoA gains. On QuALITY, a multiple-choice comprehension task, the impact is also severe, with accuracy plummeting by a staggering 48.5%. Inspired in part by the historical Doge of Venice voting process, designed to minimize influence and deception, we propose a range of unsupervised defense mechanisms that recover most of the lost performance.