Diverse Evidence, Better Forecasts: Multi-Agent Deliberation Under Information Asymmetry
For practitioners of LLM-based forecasting, this work identifies and solves a fundamental design flaw in multi-agent deliberation, showing that input diversity is the key enabler of effective reasoning.
Multi-agent LLM forecasting suffers when all agents share identical evidence, leading to herding. The authors propose InfoDelphi, which partitions evidence into shared and private subsets to create information asymmetry, achieving 12–18% Brier score improvement and 4–8 percentage point accuracy gains over baselines on the PolyGym benchmark.
Multi-agent systems are increasingly used for forecasting future events, as deliberation among multiple LLMs is believed to improve reasoning and calibration. Yet existing approaches overlook a critical design choice: what information each agent receives. When all agents are given identical evidence, deliberation collapses into herding rather than genuine belief revision, leaving multi-agent systems little better than a single agent. We identify this as a fundamental gap and propose designed information asymmetry to close it: by partitioning evidence into shared public and disjoint private subsets, each agent holds exclusive knowledge that can only reach others through deliberation. We theoretically show that this decomposition reduces inter-agent error correlation, and instantiate it in InfoDelphi, a framework combining relevance-aware evidence routing, rationale-based iterative deliberation, and confidence-weighted aggregation. On PolyGym, a benchmark of 375 binary forecasting questions derived from real-world prediction markets, InfoDelphi outperforms the strongest single-agent and multi-agent baselines by 12--18% in Brier score and 4--8 percentage points in accuracy. More detailed experiments confirm that removing information asymmetry eliminates most deliberation gains, establishing diversity of input as the key enabler of effective multi-agent reasoning.