Uncertainty-Aware Explanations Through Probabilistic Self-Explainable Neural Networks
This work addresses the need for reliable and explainable AI in high-stakes domains, offering an incremental improvement by adding uncertainty awareness to existing prototype-based methods.
The paper tackles the lack of transparency in Deep Neural Networks for high-stakes applications by introducing Prob-PSENN, a probabilistic reformulation of Prototype-Based Self-Explainable Neural Networks that replaces point estimates with probability distributions over prototypes, resulting in more meaningful and robust explanations while maintaining competitive predictive performance.
The lack of transparency of Deep Neural Networks continues to be a limitation that severely undermines their reliability and usage in high-stakes applications. Promising approaches to overcome such limitations are Prototype-Based Self-Explainable Neural Networks (PSENNs), whose predictions rely on the similarity between the input at hand and a set of prototypical representations of the output classes, offering therefore a deep, yet transparent-by-design, architecture. In this paper, we introduce a probabilistic reformulation of PSENNs, called Prob-PSENN, which replaces point estimates for the prototypes with probability distributions over their values. This provides not only a more flexible framework for an end-to-end learning of prototypes, but can also capture the explanatory uncertainty of the model, which is a missing feature in previous approaches. In addition, since the prototypes determine both the explanation and the prediction, Prob-PSENNs allow us to detect when the model is making uninformed or uncertain predictions, and to obtain valid explanations for them. Our experiments demonstrate that Prob-PSENNs provide more meaningful and robust explanations than their non-probabilistic counterparts, while remaining competitive in terms of predictive performance, thus enhancing the explainability and reliability of the models.