An Optimization Method for Autoregressive Time Series Forecasting
This addresses the issue of inefficient long-term forecasting in time series models for researchers and practitioners, offering a significant improvement over existing methods.
The paper tackles the problem of autoregressive time series forecasting by proposing a novel training method that enforces temporal causality and penalizes violations, resulting in a new state-of-the-art with over 10% MSE reduction compared to baselines and enabling reliable long-term predictions at horizons over 7.5 times longer.
Current time-series forecasting models are primarily based on transformer-style neural networks. These models achieve long-term forecasting mainly by scaling up the model size rather than through genuinely autoregressive (AR) rollout. From the perspective of large language model training, the traditional training process for time-series forecasting models ignores temporal causality. In this paper, we propose a novel training method for time-series forecasting that enforces two key properties: (1) AR prediction errors should increase with the forecasting horizon. Any violation of this principle is considered random guessing and is explicitly penalized in the loss function, and (2) the method enables models to concatenate short-term AR predictions for forming flexible long-term forecasts. Empirical results demonstrate that our method establishes a new state-of-the-art across multiple benchmarks, achieving an MSE reduction of more than 10% compared to iTransformer and other recent strong baselines. Furthermore, it enables short-horizon forecasting models to perform reliable long-term predictions at horizons over 7.5 times longer. Code is available at https://github.com/LizhengMathAi/AROpt