Identifying Drivers of Predictive Aleatoric Uncertainty
This work addresses the need for transparent uncertainty quantification in AI, which is crucial for trustable decision-making, though it is incremental as it builds on existing explainability techniques.
The authors tackled the problem of explaining predictive aleatoric uncertainty in AI models by proposing a straightforward method that adapts neural networks with Gaussian outputs and uses existing explainers, demonstrating it outperforms more complex approaches in synthetic and real datasets with quantitative benchmarks.
Explainability and uncertainty quantification are key to trustable artificial intelligence. However, the reasoning behind uncertainty estimates is generally left unexplained. Identifying the drivers of uncertainty complements explanations of point predictions in recognizing model limitations and enhancing transparent decision-making. So far, explanations of uncertainties have been rarely studied. The few exceptions rely on Bayesian neural networks or technically intricate approaches, such as auxiliary generative models, thereby hindering their broad adoption. We propose a straightforward approach to explain predictive aleatoric uncertainties. We estimate uncertainty in regression as predictive variance by adapting a neural network with a Gaussian output distribution. Subsequently, we apply out-of-the-box explainers to the model's variance output. This approach can explain uncertainty influences more reliably than complex published approaches, which we demonstrate in a synthetic setting with a known data-generating process. We substantiate our findings with a nuanced, quantitative benchmark including synthetic and real, tabular and image datasets. For this, we adapt metrics from conventional XAI research to uncertainty explanations. Overall, the proposed method explains uncertainty estimates with little modifications to the model architecture and outperforms more intricate methods in most settings.