Calibeating Made Simple
This work provides a simplified and generalized approach to calibeating, advancing online learning theory with new optimal rates and algorithms, though it is incremental in building on prior work.
The paper tackles the problem of calibeating, which involves post-processing external forecasts online to minimize cumulative losses and match an informativeness-based benchmark, by reducing it to existing online learning techniques and obtaining optimal rates for general proper losses, including new results for mixable and bounded losses, and achieving simultaneous calibration and optimal calibeating for binary predictions.
We study calibeating, the problem of post-processing external forecasts online to minimize cumulative losses and match an informativeness-based benchmark. Unlike prior work, which analyzed calibeating for specific losses with specific arguments, we reduce calibeating to existing online learning techniques and obtain results for general proper losses. More concretely, we first show that calibeating is minimax-equivalent to regret minimization. This recovers the $O(\log T)$ calibeating rate of Foster and Hart [FH23] for the Brier and log losses and its optimality, and yields new optimal calibeating rates for mixable losses and general bounded losses. Second, we prove that multi-calibeating is minimax-equivalent to the combination of calibeating and the classical expert problem. This yields new optimal multi-calibeating rates for mixable losses, including Brier and log losses, and general bounded losses. Finally, we obtain new bounds for achieving calibeating and calibration simultaneously for the Brier loss. For binary predictions, our result gives the first calibrated algorithm that at the same time also achieves the optimal $O(\log T)$ calibeating rate.