Bayesian Consensus: Consensus Estimates from Miscalibrated Instruments under Heteroscedastic Noise
This addresses the need for more reliable consensus estimates in forecasting applications like public health and finance, representing an incremental improvement over prior methods.
The paper tackles the problem of aggregating predictions from miscalibrated instruments with heteroscedastic noise by proposing a Bayesian consensus estimator, which is shown to be unbiased and asymptotically more efficient, and demonstrates improved accuracy in forecasting influenza-like illness percentages and company earnings compared to existing models.
We consider the problem of aggregating predictions or measurements from a set of human forecasters, models, sensors or other instruments which may be subject to bias or miscalibration and random heteroscedastic noise. We propose a Bayesian consensus estimator that adjusts for miscalibration and noise and show that this estimator is unbiased and asymptotically more efficient than naive alternatives. We further propose a Hierarchical Bayesian Model that leverages our proposed estimator and apply it to two real world forecasting challenges that require consensus estimates from error prone individual estimates: forecasting influenza like illness (ILI) weekly percentages and forecasting annual earnings of public companies. We demonstrate that our approach is effective at mitigating bias and error and results in more accurate forecasts than existing consensus models.