Generative Exaggeration in LLM Social Agents: Consistency, Bias, and Toxicity
This challenges the reliability of LLMs as social proxies for applications like content moderation and policy modeling, highlighting structural biases.
The study investigated how LLMs simulate political discourse on social media using data from the 2024 U.S. presidential election, finding that richer contextualization improved consistency but amplified polarization, toxicity, and a distortion called 'generation exaggeration' beyond empirical baselines.
We investigate how Large Language Models (LLMs) behave when simulating political discourse on social media. Leveraging 21 million interactions on X during the 2024 U.S. presidential election, we construct LLM agents based on 1,186 real users, prompting them to reply to politically salient tweets under controlled conditions. Agents are initialized either with minimal ideological cues (Zero Shot) or recent tweet history (Few Shot), allowing one-to-one comparisons with human replies. We evaluate three model families (Gemini, Mistral, and DeepSeek) across linguistic style, ideological consistency, and toxicity. We find that richer contextualization improves internal consistency but also amplifies polarization, stylized signals, and harmful language. We observe an emergent distortion that we call "generation exaggeration": a systematic amplification of salient traits beyond empirical baselines. Our analysis shows that LLMs do not emulate users, they reconstruct them. Their outputs, indeed, reflect internal optimization dynamics more than observed behavior, introducing structural biases that compromise their reliability as social proxies. This challenges their use in content moderation, deliberative simulations, and policy modeling.