Network Effects and Agreement Drift in LLM Debates
For researchers using LLMs to simulate social systems, this work highlights a critical bias that undermines the validity of such simulations, especially in contexts involving minority groups.
The paper investigates how LLM agents behave in multi-round debates under controlled network homophily and class sizes, revealing a directional bias called 'agreement drift' where agents shift toward specific opinions. The results caution against using LLM populations as direct proxies for human groups without accounting for structural effects and model biases.
Large Language Models (LLMs) have demonstrated an unprecedented ability to simulate human-like social behaviors, making them useful tools for simulating complex social systems. However, it remains unclear to what extent these simulations can be trusted to accurately capture key social mechanisms, particularly in highly unbalanced contexts involving minority groups. This paper uses a network generation model with controlled homophily and class sizes to examine how LLM agents behave collectively in multi-round debates. Moreover, our findings highlight a particular directional susceptibility that we term \textit{agreement drift}, in which agents are more likely to shift toward specific positions on the opinion scale. Overall, our findings highlight the need to disentangle structural effects from model biases before treating LLM populations as behavioral proxies for human groups.