On Feature Relevance Uncertainty: A Monte Carlo Dropout Sampling Approach
This addresses the need for interpretability in neural networks for real-world applications, but it is incremental as it builds on existing feature-based explanation techniques.
The paper tackles the problem of estimating uncertainty in feature relevance for neural network predictions, introducing Monte Carlo Relevance Propagation (MCRP) to compute feature relevance uncertainty scores, enabling a deeper understanding of the network's perception and reasoning.
Understanding decisions made by neural networks is key for the deployment of intelligent systems in real world applications. However, the opaque decision making process of these systems is a disadvantage where interpretability is essential. Many feature-based explanation techniques have been introduced over the last few years in the field of machine learning to better understand decisions made by neural networks and have become an important component to verify their reasoning capabilities. However, existing methods do not allow statements to be made about the uncertainty regarding a feature's relevance for the prediction. In this paper, we introduce Monte Carlo Relevance Propagation (MCRP) for feature relevance uncertainty estimation. A simple but powerful method based on Monte Carlo estimation of the feature relevance distribution to compute feature relevance uncertainty scores that allow a deeper understanding of a neural network's perception and reasoning.