Neural State-Space Models: Empirical Evaluation of Uncertainty Quantification
This addresses the need for reliable uncertainty estimation in deep learning for mission-critical applications involving non-linear dynamical systems, but it is incremental as it builds on existing Bayesian methods.
The paper tackled the problem of uncertainty quantification for neural state-space models in system identification, presenting a Bayesian approach that constructs credible intervals and a surprise index to diagnose out-of-distribution usage.
Effective quantification of uncertainty is an essential and still missing step towards a greater adoption of deep-learning approaches in different applications, including mission-critical ones. In particular, investigations on the predictive uncertainty of deep-learning models describing non-linear dynamical systems are very limited to date. This paper is aimed at filling this gap and presents preliminary results on uncertainty quantification for system identification with neural state-space models. We frame the learning problem in a Bayesian probabilistic setting and obtain posterior distributions for the neural network's weights and outputs through approximate inference techniques. Based on the posterior, we construct credible intervals on the outputs and define a surprise index which can effectively diagnose usage of the model in a potentially dangerous out-of-distribution regime, where predictions cannot be trusted.