Trust and Reliance in Consensus-Based Explanations from an Anti-Misinformation Agent
This work addresses the problem of misinformation on social media for users and designers, but it is incremental as it builds on existing XAI concepts without major breakthroughs.
The study investigated how trust and reliance affect the illusion of consensus in an AI agent designed to combat misinformation on social media, finding that reliance influences consensus-based explanations but trust does not.
The illusion of consensus occurs when people believe there is consensus across multiple sources, but the sources are the same and thus there is no "true" consensus. We explore this phenomenon in the context of an AI-based intelligent agent designed to augment metacognition on social media. Misinformation, especially on platforms like Twitter, is a global problem for which there is currently no good solution. As an explainable AI (XAI) system, the agent provides explanations for its decisions on the misinformed nature of social media content. In this late-breaking study, we explored the roles of trust (attitude) and reliance (behaviour) as key elements of XAI user experience (UX) and whether these influenced the illusion of consensus. Findings show no effect of trust, but an effect of reliance on consensus-based explanations. This work may guide the design of anti-misinformation systems that use XAI, especially the user-centred design of explanations.