Robust Aggregation of Calibrated Forecasts
For decision-makers relying on multiple calibrated forecasts, this work provides a principled aggregation method and benchmark that accounts for the information in joint distributions, clarifying the value of knowing experts' information structures.
The paper develops a framework for aggregating probabilistic forecasts from multiple experts known only to be calibrated, showing that the joint distribution of such forecasts contains decision-relevant information beyond any single expert. It introduces a robust max-min benchmark that dominates the optimal-in-hindsight benchmark and provides online algorithms to attain it.
Decision-makers often rely on multiple probabilistic forecasts that are individually calibrated but need not be fully informative. We develop a framework for aggregating such forecasts when the decision-maker knows only that experts satisfy calibration. We show that the joint distribution of calibrated forecasts can contain decision-relevant information that is unavailable from any single expert, so the standard optimal-in-hindsight (OIH) benchmark may substantially understate attainable performance. To formalize this idea, we introduce a robust max-min benchmark: the best payoff a decision-maker can guarantee against all profile-wise conditional-mean mappings compatible with calibration. This benchmark is tractable, admits a linear-programming formulation, and dominates the OIH benchmark up to calibration error. It can nevertheless be strictly below the Bayesian benchmark, clarifying the value of knowing experts' information structures. Finally, we provide online algorithms that attain the robust benchmark under forecast-only feedback and stronger contextual benchmarks under state feedback.