Robust Neural Regression via Uncertainty Learning
This addresses uncertainty estimation in neural networks, which is crucial for reliable predictions in applications like healthcare or autonomous systems, but it appears incremental as it builds on established regression techniques.
The paper tackles the problem of deep neural networks underestimating uncertainty by proposing a simple method based on iterative reweighted least squares, using two sub-networks for prediction and uncertainty estimation, which is easier to implement and more robust than existing approaches like MC Dropout or SDE-Net.
Deep neural networks tend to underestimate uncertainty and produce overly confident predictions. Recently proposed solutions, such as MC Dropout and SDENet, require complex training and/or auxiliary out-of-distribution data. We propose a simple solution by extending the time-tested iterative reweighted least square (IRLS) in generalised linear regression. We use two sub-networks to parametrise the prediction and uncertainty estimation, enabling easy handling of complex inputs and nonlinear response. The two sub-networks have shared representations and are trained via two complementary loss functions for the prediction and the uncertainty estimates, with interleaving steps as in a cooperative game. Compared with more complex models such as MC-Dropout or SDE-Net, our proposed network is simpler to implement and more robust (insensitive to varying aleatoric and epistemic uncertainty).