Auditing the Fairness of the US COVID-19 Forecast Hub's Case Prediction Models
This work addresses fairness issues in public health forecasting models used by the CDC, highlighting potential harms to specific social groups, though it is incremental as it applies existing fairness auditing methods to new data.
The study conducted a fairness analysis of the US COVID-19 Forecast Hub's case prediction models, revealing statistically significant disparities in predictive performance across social determinants like race and urbanization, with higher errors for minority groups and less urbanized areas.
The US COVID-19 Forecast Hub, a repository of COVID-19 forecasts from over 50 independent research groups, is used by the Centers for Disease Control and Prevention (CDC) for their official COVID-19 communications. As such, the Forecast Hub is a critical centralized resource to promote transparent decision making. While the Forecast Hub has provided valuable predictions focused on accuracy, there is an opportunity to evaluate model performance across social determinants such as race and urbanization level that have been known to play a role in the COVID-19 pandemic. In this paper, we carry out a comprehensive fairness analysis of the Forecast Hub model predictions and we show statistically significant diverse predictive performance across social determinants, with minority racial and ethnic groups as well as less urbanized areas often associated with higher prediction errors. We hope this work will encourage COVID-19 modelers and the CDC to report fairness metrics together with accuracy, and to reflect on the potential harms of the models on specific social groups and contexts.