Pay What LLM Wants: Can LLM Simulate Economics Experiment with 522 Real-human Persona?
This provides the first comprehensive evaluation of LLMs' capabilities in simulating economic behavior with real human data, offering empirical guidance for persona-based simulation in computational social science, though it is incremental in scope.
The study evaluated whether large language models (LLMs) could predict individual economic decision-making in Pay-What-You-Want pricing experiments using real data from 522 human personas, finding that LLMs struggled with individual-level predictions but showed reasonable group-level behavioral tendencies.
Recent advances in Large Language Models (LLMs) have generated significant interest in their capacity to simulate human-like behaviors, yet most studies rely on fictional personas rather than actual human data. We address this limitation by evaluating LLMs' ability to predict individual economic decision-making using Pay-What-You-Want (PWYW) pricing experiments with real 522 human personas. Our study systematically compares three state-of-the-art multimodal LLMs using detailed persona information from 522 Korean participants in cultural consumption scenarios. We investigate whether LLMs can accurately replicate individual human choices and how persona injection methods affect prediction performance. Results reveal that while LLMs struggle with precise individual-level predictions, they demonstrate reasonable group-level behavioral tendencies. Also, we found that commonly adopted prompting techniques are not much better than naive prompting methods; reconstruction of personal narrative nor retrieval augmented generation have no significant gain against simple prompting method. We believe that these findings can provide the first comprehensive evaluation of LLMs' capabilities on simulating economic behavior using real human data, offering empirical guidance for persona-based simulation in computational social science.