What Are You Hiding? Algorithmic Transparency and User Perceptions
This addresses the problem of how transparency affects user trust in AI systems, particularly in emotion prediction, but it is incremental as it unifies existing contradictory research.
The paper tackles the mixed impact of algorithmic transparency on user perceptions by examining an interactive emotion prediction system, showing that transparency can negatively affect accuracy perceptions for users whose expectations are not violated but limits damage when expectations are violated.
Extensive recent media focus has been directed towards the dark side of intelligent systems, how algorithms can influence society negatively. Often, transparency is proposed as a solution or step in the right direction. Unfortunately, research is mixed on the impact of transparency on the user experience. We examine transparency in the context an interactive system that predicts positive/negative emotion from a users' written text. We unify seemingly this contradictory research under a single model. We show that transparency can negatively affect accuracy perceptions for users whose expectations were not violated by the system's prediction; however, transparency also limits the damage done when users' expectations are violated by system predictions.