Informative Neural Ensemble Kalman Learning
This addresses the need for more informative learning methods in neural networks, though it appears incremental as it adapts existing Ensemble Kalman Filter techniques to neural learning.
The authors tackled the problem of developing informative learning approaches for neural networks by proposing Informative Ensemble Kalman Learning, which replaces backpropagation with an adaptive Ensemble Kalman Filter to quantify uncertainty and maximize information gain during learning, showing competitive performance on standard datasets and recovering dynamical equations from Lorenz-63 system simulations.
In stochastic systems, informative approaches select key measurement or decision variables that maximize information gain to enhance the efficacy of model-related inferences. Neural Learning also embodies stochastic dynamics, but informative Learning is less developed. Here, we propose Informative Ensemble Kalman Learning, which replaces backpropagation with an adaptive Ensemble Kalman Filter to quantify uncertainty and enables maximizing information gain during Learning. After demonstrating Ensemble Kalman Learning's competitive performance on standard datasets, we apply the informative approach to neural structure learning. In particular, we show that when trained from the Lorenz-63 system's simulations, the efficaciously learned structure recovers the dynamical equations. To the best of our knowledge, Informative Ensemble Kalman Learning is new. Results suggest that this approach to optimized Learning is promising.