Causal Perception in Question-Answering Systems
This research addresses the critical problem of misleading causal claims in question-answering systems for general users, highlighting how visual aids and warnings impact perception.
This paper investigates how question-answering systems, when providing answers to "why" questions, can mislead users with questionable causal claims. Through two crowdsourced experiments, the authors found that displaying a scatterplot increased the perceived plausibility of unreasonable causal claims, while a "correlation is not causation" warning made users more cautious about accepting reasonable causal claims.
Root cause analysis is a common data analysis task. While question-answering systems enable people to easily articulate a why question (e.g., why students in Massachusetts have high ACT Math scores on average) and obtain an answer, these systems often produce questionable causal claims. To investigate how such claims might mislead users, we conducted two crowdsourced experiments to study the impact of showing different information on user perceptions of a question-answering system. We found that in a system that occasionally provided unreasonable responses, showing a scatterplot increased the plausibility of unreasonable causal claims. Also, simply warning participants that correlation is not causation seemed to lead participants to accept reasonable causal claims more cautiously. We observed a strong tendency among participants to associate correlation with causation. Yet, the warning appeared to reduce the tendency. Grounded in the findings, we propose ways to reduce the illusion of causality when using question-answering systems.