The Qualitative Laboratory: Theory Prototyping and Hypothesis Generation with Large Language Models
This addresses the problem of hypothesis generation for social scientists by offering a tool to overcome limitations in existing methods like vignette surveys and agent-based models, though it is incremental in applying LLMs to this specific domain.
The paper tackles the challenge of generating qualitative hypotheses in social science by introducing a novel method using Large Language Models (LLMs) to simulate sociological personas, which produced nuanced and counter-intuitive hypotheses, such as a conservative persona rejecting a national security frame.
A central challenge in social science is to generate rich qualitative hypotheses about how diverse social groups might interpret new information. This article introduces and illustrates a novel methodological approach for this purpose: sociological persona simulation using Large Language Models (LLMs), which we frame as a "qualitative laboratory". We argue that for this specific task, persona simulation offers a distinct advantage over established methods. By generating naturalistic discourse, it overcomes the lack of discursive depth common in vignette surveys, and by operationalizing complex worldviews through natural language, it bypasses the formalization bottleneck of rule-based agent-based models (ABMs). To demonstrate this potential, we present a protocol where personas derived from a sociological theory of climate reception react to policy messages. The simulation produced nuanced and counter-intuitive hypotheses - such as a conservative persona's rejection of a national security frame - that challenge theoretical assumptions. We conclude that this method, used as part of a "simulation then validation" workflow, represents a superior tool for generating deeply textured hypotheses for subsequent empirical testing.