The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspective
This addresses a critical problem for researchers using LLMs to simulate human behavior in experiments, highlighting a unique challenge that requires new design approaches rather than incremental adaptations.
The paper identifies a fundamental challenge in using LLMs to simulate human behavior in experiments: when LLMs are blind to experimental designs, treatment variations can violate unconfoundedness assumptions, leading to implausible results, as shown in a demand estimation context with an actual experiment benchmark. It formalizes this as a trade-off between unconfoundedness and ecological validity, stemming from ambiguous prompting strategies that cannot be fixed by data improvements, and suggests rethinking experimental design practices for LLMs.
Large Language Models (LLMs) have shown impressive potential to simulate human behavior. We identify a fundamental challenge in using them to simulate experiments: when LLM-simulated subjects are blind to the experimental design (as is standard practice with human subjects), variations in treatment systematically affect unspecified variables that should remain constant, violating the unconfoundedness assumption. Using demand estimation as a context and an actual experiment as a benchmark, we show this can lead to implausible results. While confounding may in principle be addressed by controlling for covariates, this can compromise ecological validity in the context of LLM simulations: controlled covariates become artificially salient in the simulated decision process, which introduces focalism. This trade-off between unconfoundedness and ecological validity is usually absent in traditional experimental design and represents a unique challenge in LLM simulations. We formalize this challenge theoretically, showing it stems from ambiguous prompting strategies, and hence cannot be fully addressed by improving training data or by fine-tuning. Alternative approaches that unblind the experimental design to the LLM show promise. Our findings suggest that effectively leveraging LLMs for experimental simulations requires fundamentally rethinking established experimental design practices rather than simply adapting protocols developed for human subjects.