Spectral Predictability as a Fast Reliability Indicator for Time Series Forecasting Model Selection
This provides a practical tool for practitioners to reduce validation costs in time series forecasting, though it is incremental as it builds on existing signal processing metrics.
The paper tackles the problem of computationally expensive model selection in time series forecasting by introducing spectral predictability (Ω) as a fast reliability indicator, showing that it stratifies model performance across 51 models and 28 datasets, with large foundation models outperforming simpler ones when Ω is high but losing advantage as Ω drops.
Practitioners deploying time series forecasting models face a dilemma: exhaustively validating dozens of models is computationally prohibitive, yet choosing the wrong model risks poor performance. We show that spectral predictability~$Ω$ -- a simple signal processing metric -- systematically stratifies model family performance, enabling fast model selection. We conduct controlled experiments in four different domains, then further expand our analysis to 51 models and 28 datasets from the GIFT-Eval benchmark. We find that large time series foundation models (TSFMs) systematically outperform lightweight task-trained baselines when $Ω$ is high, while their advantage vanishes as $Ω$ drops. Computing $Ω$ takes seconds per dataset, enabling practitioners to quickly assess whether their data suits TSFM approaches or whether simpler, cheaper models suffice. We demonstrate that $Ω$ stratifies model performance predictably, offering a practical first-pass filter that reduces validation costs while highlighting the need for models that excel on genuinely difficult (low-$Ω$) problems rather than merely optimizing easy ones.