Failure of Calibration is Typical
This work addresses foundational issues in probability and statistics, highlighting potential limitations in Bayesian methods for forecasting.
The paper strengthens a prior result by showing that, from a topological perspective, calibration failure is typical and calibration rare for forecasting systems, while Bayesian forecasters remain certain of their calibration, raising concerns about Bayesian rationality.
Schervish (1985b) showed that every forecasting system is noncalibrated for uncountably many data sequences that it might see. This result is strengthened here: from a topological point of view, failure of calibration is typical and calibration rare. Meanwhile, Bayesian forecasters are certain that they are calibrated---this invites worries about the connection between Bayesianism and rationality.