Stability of the Decoupled Extended Kalman Filter Learning Algorithm in LSTM-Based Online Learning
This work addresses stability issues in online learning for LSTM networks, which is incremental as it builds on existing DEKF methods by providing theoretical conditions.
The paper tackles the convergence and stability of the decoupled extended Kalman filter (DEKF) algorithm in LSTM-based online learning, showing that DEKF achieves similar convergence and stability as the global extended Kalman filter if perturbations remain bounded, with verification through numerical simulations.
We investigate the convergence and stability properties of the decoupled extended Kalman filter learning algorithm (DEKF) within the long-short term memory network (LSTM) based online learning framework. For this purpose, we model DEKF as a perturbed extended Kalman filter and derive sufficient conditions for its stability during LSTM training. We show that if the perturbations -- introduced due to decoupling -- stay bounded, DEKF learns LSTM parameters with similar convergence and stability properties of the global extended Kalman filter learning algorithm. We verify our results with several numerical simulations and compare DEKF with other LSTM training methods. In our simulations, we also observe that the well-known hyper-parameter selection approaches used for DEKF in the literature satisfy our conditions.