A Crowdsourced Study of ChatBot Influence in Value-Driven Decision Making Scenarios
This reveals risks of manipulative uses of LLMs in decision-making scenarios like healthcare and finance, distinct from overt bias or misinformation, though it is incremental as it builds on prior work on persuasion.
The study tested whether value-framing alone in LLM-based ChatBots can influence users' decisions, finding that 336 participants significantly changed their budget choices in a defense spending scenario when exposed to value-framed ChatBots, with some showing a backfire effect when frames misaligned with their values.
Similar to social media bots that shape public opinion, healthcare and financial decisions, LLM-based ChatBots like ChatGPT can persuade users to alter their behavior. Unlike prior work that persuades via overt-partisan bias or misinformation, we test whether framing alone suffices. We conducted a crowdsourced study, where 336 participants interacted with a neutral or one of two value-framed ChatBots while deciding to alter US defense spending. In this single policy domain with controlled content, participants exposed to value-framed ChatBots significantly changed their budget choices relative to the neutral control. When the frame misaligned with their values, some participants reinforced their original preference, revealing a potentially replicable backfire effect, originally considered rare in the literature. These findings suggest that value-framing alone lowers the barrier for manipulative uses of LLMs, revealing risks distinct from overt bias or misinformation, and clarifying risks to countering misinformation.