Revealing Political Bias in LLMs through Structured Multi-Agent Debate
This study addresses political bias in LLMs for social simulation applications, though it is incremental as it builds on existing multi-agent debate research.
The researchers investigated political bias in LLMs using a structured multi-agent debate framework with Neutral, Republican, and Democrat agents, finding that Neutral agents consistently aligned with Democrats, Republicans shifted toward Neutral, gender influenced attitudes, and agents with shared affiliations formed echo chambers that intensified attitudes.
Large language models (LLMs) are increasingly used to simulate social behaviour, yet their political biases and interaction dynamics in debates remain underexplored. We investigate how LLM type and agent gender attributes influence political bias using a structured multi-agent debate framework, by engaging Neutral, Republican, and Democrat American LLM agents in debates on politically sensitive topics. We systematically vary the underlying LLMs, agent genders, and debate formats to examine how model provenance and agent personas influence political bias and attitudes throughout debates. We find that Neutral agents consistently align with Democrats, while Republicans shift closer to the Neutral; gender influences agent attitudes, with agents adapting their opinions when aware of other agents' genders; and contrary to prior research, agents with shared political affiliations can form echo chambers, exhibiting the expected intensification of attitudes as debates progress.