Show or Suppress? Managing Input Uncertainty in Machine Learning Model Explanations
This work addresses the challenge of communicating uncertainty in model interpretability for users, but it is incremental as it builds on existing feature attribution methods.
The paper tackles the problem of how input uncertainty affects trust in machine learning model explanations, proposing two approaches to manage uncertainty perception: showing uncertainty in feature attributions or suppressing it with regularization. Through experiments and user evaluations, they found benefits in moderately suppressing attribution uncertainty and concerns about showing it.
Feature attribution is widely used in interpretable machine learning to explain how influential each measured input feature value is for an output inference. However, measurements can be uncertain, and it is unclear how the awareness of input uncertainty can affect the trust in explanations. We propose and study two approaches to help users to manage their perception of uncertainty in a model explanation: 1) transparently show uncertainty in feature attributions to allow users to reflect on, and 2) suppress attribution to features with uncertain measurements and shift attribution to other features by regularizing with an uncertainty penalty. Through simulation experiments, qualitative interviews, and quantitative user evaluations, we identified the benefits of moderately suppressing attribution uncertainty, and concerns regarding showing attribution uncertainty. This work adds to the understanding of handling and communicating uncertainty for model interpretability.