Wisdom Of The (AI) Crowd: Investigating Artificial Swarm Intelligence In Large Language Models
For organizations using LLMs in decision-making, this work demonstrates that aggregating multiple LLM outputs can significantly improve accuracy, but the effect is incremental over existing ensemble methods.
The paper investigates whether LLMs can replicate human swarm intelligence effects through intra- and inter-model aggregation, finding consistent error reductions up to 37 percentage points in MAPE across eight estimation tasks.
Human swarm intelligence demonstrates remarkable collective accuracy but faces scalability constraints in cost, coordination, and time. We investigate whether large language models (LLMs) can approximate swarm intelligence effects through artificial swarms, addressing a critical gap in understanding AI-based aggregation mechanisms. We conducted a controlled experiment with 960 manually executed prompts across three proprietary models (GPT-5, Gemini 2.5 Pro, Claude Sonnet 4.5), testing intra-model sampling and inter-model aggregation on eight estimation tasks. Results reveal consistent error reduction through intra- and inter-model aggregation, with significant error reductions up to 37 percentage points in MAPE across different aggregation strategies. We observed small to large effect sizes for positive correlations (Spearman's $ρ=0.242-0.568$, all $p<0.001$) between relative confidence interval widths and relative estimation errors, suggesting LLMs possess metacognitive awareness when assessing uncertainty. We discuss implications for research and practice, providing actionable insights for deploying LLM swarms in organizational decision-making.