To Trust or Not to Trust a Regressor: Estimating and Explaining Trustworthiness of Regression Predictions
This addresses the problem of trust assessment in hybrid human-AI systems for users, but it is incremental as it builds on existing work in explainable AI and trust estimation.
The paper tackles the problem of users needing to decide whether to trust algorithmic predictions in hybrid human-AI systems by introducing RETRO-VIZ, a method that estimates and explains trustworthiness of regression predictions. The results show that RETRO-scores negatively correlate with prediction error across 117 settings, and in a user study with 41 participants, VIZ-explanations helped 95.1% correctly select the more trustworthy prediction and 75.6% accurately describe the reasons.
In hybrid human-AI systems, users need to decide whether or not to trust an algorithmic prediction while the true error in the prediction is unknown. To accommodate such settings, we introduce RETRO-VIZ, a method for (i) estimating and (ii) explaining trustworthiness of regression predictions. It consists of RETRO, a quantitative estimate of the trustworthiness of a prediction, and VIZ, a visual explanation that helps users identify the reasons for the (lack of) trustworthiness of a prediction. We find that RETRO-scores negatively correlate with prediction error across 117 experimental settings, indicating that RETRO provides a useful measure to distinguish trustworthy predictions from untrustworthy ones. In a user study with 41 participants, we find that VIZ-explanations help users identify whether a prediction is trustworthy or not: on average, 95.1% of participants correctly select the more trustworthy prediction, given a pair of predictions. In addition, an average of 75.6% of participants can accurately describe why a prediction seems to be (not) trustworthy. Finally, we find that the vast majority of users subjectively experience RETRO-VIZ as a useful tool to assess the trustworthiness of algorithmic predictions.