Deep Learning with Kernel Flow Regularization for Time Series Forecasting
This is an incremental improvement for time series forecasting practitioners, addressing overfitting in LSTMs.
The paper tackles overfitting in LSTM neural networks for time series forecasting by applying kernel flow regularization, showing it outperforms baselines and achieves similar effects to dropout, with best results combining both methods on datasets like power-load demand forecasts.
Long Short-Term Memory (LSTM) neural networks have been widely used for time series forecasting problems. However, LSTMs are prone to overfitting and performance reduction during test phases. Several different regularization techniques have been shown in literature to prevent overfitting problems in neural networks. In this paper, first, we introduce application of kernel flow methods for time series forecasting in general. Afterward, we examine the effectiveness of applying kernel flow regularization on LSTM layers to avoid overfitting problems. We describe a regularization method by applying kernel flow loss function on LSTM layers. In experimental results, we show that kernel flow outperforms baseline models on time series forecasting benchmarks. We also compare the effect of dropout and kernel flow regularization techniques on LSTMs. The experimental results illustrate that kernel flow achieves similar regularization effect to dropout. It also shows that the best results is obtained using both kernel flow and dropout regularizations with early stopping on LSTM layers on some time series datasets (e.g. power-load demand forecasts).