When Helping Hurts and How to Fix It: Multi-Agent Debate for Data Cleaning
For practitioners of LLM-based data cleaning, this work identifies when multi-agent debate helps or hurts and provides a principled condition to predict its effectiveness.
Multi-agent debate for data cleaning can hurt generation (up to -15.5pp) due to hallucinated Critic feedback, but improves error detection (+27.4pp F1). The authors derive a debate benefit condition and show that adversarial separation with code-execution grounding yields the first debate configuration to significantly exceed single-agent on a generative task (+5.3pp).
When does multi-agent debate help data cleaning, and when does it hurt? Across three benchmarks, four model families, and over 6,000 task-condition pairs, we find debate's effect reverses sign: it degrades generation across all four models (-1.6 to -15.5pp) through critique-induced confusion (CIC), hallucinated Critic feedback that the Generator accepts uncritically, yet improves error detection (+27.4pp F1, d=1.0). We derive a debate benefit condition: debate helps when the probability of rescuing a wrong output (Critic verification odds weighted by fixability) exceeds the probability of destroying a correct one. A factorial experiment proves adversarial separation is essential: self-verification with identical tools fails, while a separate Critic with code-execution grounding and evidence-gated generation produces the first debate configuration to significantly exceed single-agent on a generative task (+5.3pp, p<0.05). The condition correctly predicts all nine task types and generalizes with zero false positives across 19 published comparisons in seven domains.