Identity, Cooperation and Framing Effects within Groups of Real and Simulated Humans
This work addresses the challenge of accurately replicating identity-based behaviors in simulations for researchers in social sciences and AI, though it is incremental by building on prior methods like steering.
The study tackled the problem of simulating human action in social dilemma games by deeply binding base large language models with extended backstories, resulting in improved simulation fidelity compared to human studies through conditioning with narrative identities and consistency checks.
Humans act via a nuanced process that depends both on rational deliberation and also on identity and contextual factors. In this work, we study how large language models (LLMs) can simulate human action in the context of social dilemma games. While prior work has focused on "steering" (weak binding) of chat models to simulate personas, we analyze here how deep binding of base models with extended backstories leads to more faithful replication of identity-based behaviors. Our study has these findings: simulation fidelity vs human studies is improved by conditioning base LMs with rich context of narrative identities and checking consistency using instruction-tuned models. We show that LLMs can also model contextual factors such as time (year that a study was performed), question framing, and participant pool effects. LLMs, therefore, allow us to explore the details that affect human studies but which are often omitted from experiment descriptions, and which hamper accurate replication.