Calibrated Multi-Level Quantile Forecasting
This addresses the need for reliable uncertainty quantification in time-series forecasting for domains like epidemics and energy, though it is incremental as it builds on existing forecasters.
The paper tackles the problem of ensuring quantile forecasts are calibrated across multiple levels simultaneously, even under adversarial distribution shifts, and presents MultiQT, a lightweight wrapper that guarantees calibration and ordered quantiles without degrading performance, with experiments showing significant calibration improvements in epidemic and energy forecasting.
We develop an online method that guarantees calibration of quantile forecasts at multiple quantile levels simultaneously. In this work, a sequence of quantile forecasts is said to be calibrated provided that its $α$-level predictions are greater than or equal to the target value at an $α$ fraction of time steps, for each level $α$. Our procedure, called the multi-level quantile tracker (MultiQT), is lightweight and wraps around any point or quantile forecaster to produce adjusted quantile forecasts that are guaranteed to be calibrated, even against adversarial distribution shifts. Critically, it does so while ensuring that the quantiles remain ordered, e.g., the 0.5-level quantile forecast will never be larger than the 0.6-level forecast. Moreover, the method has a no-regret guarantee, implying it will not degrade the performance of the existing forecaster (asymptotically), with respect to the quantile loss. In our experiments, we find that MultiQT significantly improves the calibration of real forecasters in epidemic and energy forecasting problems, while leaving the quantile loss largely unchanged or slightly improved.