A Large-Scale Simulation on Large Language Models for Decision-Making in Political Science
This work addresses the problem of modeling voter behavior for political science researchers, offering a scalable framework but is incremental in applying LLMs to this domain.
The study tackled the challenge of simulating human decision-making in political contexts using large language models (LLMs), developing a theory-driven framework that improved simulation accuracy for U.S. presidential elections while mitigating biases.
While LLMs have demonstrated remarkable capabilities in text generation and reasoning, their ability to simulate human decision-making -- particularly in political contexts -- remains an open question. However, modeling voter behavior presents unique challenges due to limited voter-level data, evolving political landscapes, and the complexity of human reasoning. In this study, we develop a theory-driven, multi-step reasoning framework that integrates demographic, temporal and ideological factors to simulate voter decision-making at scale. Using synthetic personas calibrated to real-world voter data, we conduct large-scale simulations of recent U.S. presidential elections. Our method significantly improves simulation accuracy while mitigating model biases. We examine its robustness by comparing performance across different LLMs. We further investigate the challenges and constraints that arise from LLM-based political simulations. Our work provides both a scalable framework for modeling political decision-making behavior and insights into the promise and limitations of using LLMs in political science research.