Approaching Neural Network Uncertainty Realism
This work addresses the problem of accurately quantifying and evaluating uncertainty in neural networks for safety-critical systems like autonomous vehicles, which is an incremental improvement in the field of uncertainty quantification.
The paper proposes a Mahalanobis distance-based statistical test to evaluate uncertainty realism in regression tasks, a critical aspect for safety-critical systems. Their empirical evaluation highlights the necessity for uncertainty measures capable of upper-bounding heavy-tailed empirical errors, and they demonstrate that a variational U-Net architecture significantly improves uncertainty realism in automotive image-to-image tasks compared to a plain encoder-decoder model.
Statistical models are inherently uncertain. Quantifying or at least upper-bounding their uncertainties is vital for safety-critical systems such as autonomous vehicles. While standard neural networks do not report this information, several approaches exist to integrate uncertainty estimates into them. Assessing the quality of these uncertainty estimates is not straightforward, as no direct ground truth labels are available. Instead, implicit statistical assessments are required. For regression, we propose to evaluate uncertainty realism -- a strict quality criterion -- with a Mahalanobis distance-based statistical test. An empirical evaluation reveals the need for uncertainty measures that are appropriate to upper-bound heavy-tailed empirical errors. Alongside, we transfer the variational U-Net classification architecture to standard supervised image-to-image tasks. We adopt it to the automotive domain and show that it significantly improves uncertainty realism compared to a plain encoder-decoder model.