tsGT: Stochastic Time Series Modeling With Transformer
This addresses the problem of improving time series forecasting accuracy for researchers and practitioners, though it appears incremental by adapting transformers with stochastic elements.
The paper tackled time series modeling by introducing tsGT, a stochastic transformer model, and showed that it outperforms state-of-the-art models on metrics like MAD and RMSE, and surpasses stochastic peers on QL and CRPS across four datasets.
Time series methods are of fundamental importance in virtually any field of science that deals with temporally structured data. Recently, there has been a surge of deterministic transformer models with time series-specific architectural biases. In this paper, we go in a different direction by introducing tsGT, a stochastic time series model built on a general-purpose transformer architecture. We focus on using a well-known and theoretically justified rolling window backtesting and evaluation protocol. We show that tsGT outperforms the state-of-the-art models on MAD and RMSE, and surpasses its stochastic peers on QL and CRPS, on four commonly used datasets. We complement these results with a detailed analysis of tsGT's ability to model the data distribution and predict marginal quantile values.