Can LLMs Emulate Human Belief Dynamics?
For social science researchers using LLMs as human surrogates, this paper provides a strong negative result showing current LLMs are inadequate for simulating belief dynamics.
The paper tests whether LLMs can emulate human belief dynamics in social networks by replicating an established study across 12 models. They find LLMs systematically fail, being more conformist and misrepresenting initial belief distributions, warning against using LLMs as human proxies.
Can LLMs simulate how humans form and change beliefs in social networks? We put this to the test by replicating an established study on belief dynamics, evaluating 12 LLMs across multiple model families and parameter sizes. The answer is a clear no, and in systematic ways. LLMs fail to capture initial human belief distributions and tend to be overall more conformist than humans, shifting their responses to align with those around them. They also take a nuanced approach to emulating human homophilic tendencies within networks. Our findings carry a double payoff: they highlight fundamental properties of LLM behavior, and they raise a sharp warning against deploying LLMs as human proxies in social simulations.