Recalibrating probabilistic forecasts of epidemics
This work addresses unreliable uncertainty assignment in epidemic forecasts, providing a practical post-processing tool for public health applications, though it is incremental as it builds on existing recalibration techniques.
The authors tackled the problem of miscalibrated probabilistic forecasts in epidemics by introducing a recalibration method that improves calibration and log score performance, demonstrating its effectiveness on 27 influenza forecasters in the FluSight Network.
Distributional forecasts are important for a wide variety of applications, including forecasting epidemics. Often, forecasts are miscalibrated, or unreliable in assigning uncertainty to future events. We present a recalibration method that can be applied to a black-box forecaster given retrospective forecasts and observations, as well as an extension to make this method more effective in recalibrating epidemic forecasts. This method is guaranteed to improve calibration and log score performance when trained and measured in-sample. We also prove that the increase in expected log score of a recalibrated forecaster is equal to the entropy of the PIT distribution. We apply this recalibration method to the 27 influenza forecasters in the FluSight Network and show that recalibration reliably improves forecast accuracy and calibration. This method is effective, robust, and easy to use as a post-processing tool to improve epidemic forecasts.