Considering Nonstationary within Multivariate Time Series with Variational Hierarchical Transformer for Forecasting
This work addresses the problem of accurate forecasting for multivariate time series, which is important for applications like finance or climate modeling, by introducing a novel method that integrates probabilistic modeling with transformers, representing an incremental advancement over existing stationarization approaches.
The paper tackles the challenge of forecasting multivariate time series by addressing non-stationarity and stochasticity, proposing HTV-Trans, which combines a hierarchical probabilistic generative module with a transformer to recover intrinsic non-stationary information, achieving improved forecasting performance as demonstrated in experiments on diverse datasets.
The forecasting of Multivariate Time Series (MTS) has long been an important but challenging task. Due to the non-stationary problem across long-distance time steps, previous studies primarily adopt stationarization method to attenuate the non-stationary problem of the original series for better predictability. However, existing methods always adopt the stationarized series, which ignores the inherent non-stationarity, and has difficulty in modeling MTS with complex distributions due to the lack of stochasticity. To tackle these problems, we first develop a powerful hierarchical probabilistic generative module to consider the non-stationarity and stochastic characteristics within MTS, and then combine it with transformer for a well-defined variational generative dynamic model named Hierarchical Time series Variational Transformer (HTV-Trans), which recovers the intrinsic non-stationary information into temporal dependencies. Being a powerful probabilistic model, HTV-Trans is utilized to learn expressive representations of MTS and applied to forecasting tasks. Extensive experiments on diverse datasets show the efficiency of HTV-Trans on MTS forecasting tasks