Estimating the Uncertainty of Neural Network Forecasts for Influenza Prevalence Using Web Search Activity
This work addresses the need for uncertainty-aware forecasting in public health decision-making for influenza, though it is incremental as it builds on existing methods for neural networks and web search data.
The paper tackled the problem of forecasting influenza prevalence using web search activity by demonstrating that Bayesian Neural Networks (BNNs) can provide forecasts with uncertainty estimates without significant accuracy loss compared to traditional neural networks, showing superior accuracy for horizons greater than 7 days when modeling both data and model uncertainty.
Influenza is an infectious disease with the potential to become a pandemic, and hence, forecasting its prevalence is an important undertaking for planning an effective response. Research has found that web search activity can be used to improve influenza models. Neural networks (NN) can provide state-of-the-art forecasting accuracy but do not commonly incorporate uncertainty in their estimates, something essential for using them effectively during decision making. In this paper, we demonstrate how Bayesian Neural Networks (BNNs) can be used to both provide a forecast and a corresponding uncertainty without significant loss in forecasting accuracy compared to traditional NNs. Our method accounts for two sources of uncertainty: data and model uncertainty, arising due to measurement noise and model specification, respectively. Experiments are conducted using 14 years of data for England, assessing the model's accuracy over the last 4 flu seasons in this dataset. We evaluate the performance of different models including competitive baselines with conventional metrics as well as error functions that incorporate uncertainty estimates. Our empirical analysis indicates that considering both sources of uncertainty simultaneously is superior to considering either one separately. We also show that a BNN with recurrent layers that models both sources of uncertainty yields superior accuracy for these metrics for forecasting horizons greater than 7 days.