The effects of regularisation on RNN models for time series forecasting: Covid-19 as an example
This work addresses the problem of improving neural network performance on limited data for pandemic forecasting, but it is incremental as it applies existing regularization techniques to a specific domain.
The paper tackled the challenge of training neural networks on small datasets for COVID-19 time series forecasting by testing six regularization methods, finding that a GRU with 20% Dropout achieved the lowest RMSE scores and reduced RMSE by 23% when trained on only 28 days of data.
Many research papers that propose models to predict the course of the COVID-19 pandemic either use handcrafted statistical models or large neural networks. Even though large neural networks are more powerful than simpler statistical models, they are especially hard to train on small datasets. This paper not only presents a model with grater flexibility than the other proposed neural networks, but also presents a model that is effective on smaller datasets. To improve performance on small data, six regularisation methods were tested. The results show that the GRU combined with 20% Dropout achieved the lowest RMSE scores. The main finding was that models with less access to data relied more on the regulariser. Applying Dropout to a GRU model trained on only 28 days of data reduced the RMSE by 23%.