Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulation
This addresses the problem of overly constrained simulations in social science research, proposing a shift toward more generative methods, though it is incremental as it builds on existing survey methodology and NLP advances.
The paper argues that using open-ended, free-form text in LLM-based social simulations is essential for realism, as it captures topics, viewpoints, and reasoning processes, improving measurement, reducing bias, and supporting exploration of unanticipated views.
Large Language Models (LLMs) are increasingly used to simulate public opinion and other social phenomena. Most current studies constrain these simulations to multiple-choice or short-answer formats for ease of scoring and comparison, but such closed designs overlook the inherently generative nature of LLMs. In this position paper, we argue that open-endedness, using free-form text that captures topics, viewpoints, and reasoning processes "in" LLMs, is essential for realistic social simulation. Drawing on decades of survey-methodology research and recent advances in NLP, we argue why this open-endedness is valuable in LLM social simulations, showing how it can improve measurement and design, support exploration of unanticipated views, and reduce researcher-imposed directive bias. It also captures expressiveness and individuality, aids in pretesting, and ultimately enhances methodological utility. We call for novel practices and evaluation frameworks that leverage rather than constrain the open-ended generative diversity of LLMs, creating synergies between NLP and social science.