Persuasion at Play: Understanding Misinformation Dynamics in Demographic-Aware Human-LLM Interactions
This research addresses the problem of misinformation spread in AI-human interactions, with implications for vulnerable demographic groups, though it is incremental in building on existing knowledge of demographic biases.
The study investigated how demographic factors affect susceptibility to misinformation in interactions between humans and large language models (LLMs), finding that LLMs reflect human-like demographic patterns in misinformation susceptibility and exhibit echo chamber behavior.
Existing challenges in misinformation exposure and susceptibility vary across demographic groups, as some populations are more vulnerable to misinformation than others. Large language models (LLMs) introduce new dimensions to these challenges through their ability to generate persuasive content at scale and reinforcing existing biases. This study investigates the bidirectional persuasion dynamics between LLMs and humans when exposed to misinformative content. We analyze human-to-LLM influence using human-stance datasets and assess LLM-to-human influence by generating LLM-based persuasive arguments. Additionally, we use a multi-agent LLM framework to analyze the spread of misinformation under persuasion among demographic-oriented LLM agents. Our findings show that demographic factors influence susceptibility to misinformation in LLMs, closely reflecting the demographic-based patterns seen in human susceptibility. We also find that, similar to human demographic groups, multi-agent LLMs exhibit echo chamber behavior. This research explores the interplay between humans and LLMs, highlighting demographic differences in the context of misinformation and offering insights for future interventions.