Enabling Uncertainty Estimation in Iterative Neural Networks
This provides a low-cost uncertainty estimation method for iterative neural networks, applicable in domains like aerial imaging and aerodynamics, but it is incremental as it builds on existing iterative architectures.
The paper tackled the problem of uncertainty estimation in iterative neural networks by proposing that convergence rate correlates with accuracy, enabling state-of-the-art estimates at lower computational cost than methods like Ensembles, with demonstrations in road detection and aerodynamic property estimation.
Turning pass-through network architectures into iterative ones, which use their own output as input, is a well-known approach for boosting performance. In this paper, we argue that such architectures offer an additional benefit: The convergence rate of their successive outputs is highly correlated with the accuracy of the value to which they converge. Thus, we can use the convergence rate as a useful proxy for uncertainty. This results in an approach to uncertainty estimation that provides state-of-the-art estimates at a much lower computational cost than techniques like Ensembles, and without requiring any modifications to the original iterative model. We demonstrate its practical value by embedding it in two application domains: road detection in aerial images and the estimation of aerodynamic properties of 2D and 3D shapes.