You May Not Need Order in Time Series Forecasting
This addresses a data efficiency problem for time series forecasting practitioners, though it appears incremental as it builds on existing transformer methods.
The paper tackles the challenge of training transformers for time series forecasting with limited data by proposing a random sampling technique that breaks temporal order in training windows, achieving competitive results compared to state-of-the-art methods on real-world datasets.
Time series forecasting with limited data is a challenging yet critical task. While transformers have achieved outstanding performances in time series forecasting, they often require many training samples due to the large number of trainable parameters. In this paper, we propose a training technique for transformers that prepares the training windows through random sampling. As input time steps need not be consecutive, the number of distinct samples increases from linearly to combinatorially many. By breaking the temporal order, this technique also helps transformers to capture dependencies among time steps in finer granularity. We achieve competitive results compared to the state-of-the-art on real-world datasets.