Minority Sentinel: When to Overturn Majority Voting in Multi-Agent LLM Debates
For practitioners using multi-agent LLM systems, this work provides a lightweight method to recover suppressed correct answers without degrading overall accuracy.
The paper identifies that multi-agent LLM debates with majority voting systematically suppress correct minority opinions due to correlated errors, recovering a 10-percentage-point margin. The proposed Minority Sentinel, a LightGBM classifier using debate logs, achieves 81.2% flip precision with positive net gain across six benchmarks.
Multi-Agent Debate (MAD) with Majority Voting is a dominant paradigm for improving LLM reasoning, yet its effectiveness rests on the Condorcet Jury Theorem's assumption of independent errors. Because contemporary LLMs share similar pretraining corpora, their errors are strongly correlated, causing the majority to systematically suppress correct minority opinions, a phenomenon we term Minority Truth. Through debates among three heterogeneous LLM agents on six benchmarks, we find that roughly one in four divergent cases has the minority holding the correct answer, yielding a 10-percentage-point theoretical recovery margin. We propose Minority Sentinel, a lightweight meta-classifier that extracts a multi-dimensional debate fingerprint from debate logs and trains a LightGBM model to decide when to overturn majority voting. Minority Sentinel achieves a stable Flip Precision of 81.2% with positive Net Gain across all six datasets and all 20 random seed trials, demonstrating that debate logs contain sufficient behavioral signals for a non-LLM classifier to reliably recover suppressed minorities without degrading system accuracy. The LLM-as-Judge baseline yields negative Net Gain despite higher recall, confirming that flip safety, not recovery volume, determines intervention value.