Perceived Political Bias in LLMs Reduces Persuasive Abilities
This highlights a key limitation for using conversational AI to correct misinformation in politically polarized contexts.
The study tested whether perceived political bias reduces the persuasive ability of LLMs, finding that a warning about bias attenuated persuasion by 28% in a survey experiment with 2,144 participants.
Conversational AI has been proposed as a scalable way to correct public misconceptions and spread misinformation. Yet its effectiveness may depend on perceptions of its political neutrality. As LLMs enter partisan conflict, elites increasingly portray them as ideologically aligned. We test whether these credibility attacks reduce LLM-based persuasion. In a preregistered U.S. survey experiment (N=2144), participants completed a three-round conversation with ChatGPT about a personally held economic policy misconception. Compared to a neutral control, a short message indicating that the LLM was biased against the respondent's party attenuated persuasion by 28%. Transcript analysis indicates that the warnings alter the interaction: respondents push back more and engage less receptively. These findings suggest that the persuasive impact of conversational AI is politically contingent, constrained by perceptions of partisan alignment.