Transformer for Times Series: an Application to the S&P500
This work explores a novel application of Transformers to financial forecasting, which could benefit quantitative analysts and traders, but it appears incremental as it adapts an existing method to a new domain.
The authors applied a Transformer model to financial time series, achieving accurate next-move predictions on synthetic Ornstein-Uhlenbeck data and interesting results for S&P500 volatility prediction.
The transformer models have been extensively used with good results in a wide area of machine learning applications including Large Language Models and image generation. Here, we inquire on the applicability of this approach to financial time series. We first describe the dataset construction for two prototypical situations: a mean reverting synthetic Ornstein-Uhlenbeck process on one hand and real S&P500 data on the other hand. Then, we present in detail the proposed Transformer architecture and finally we discuss some encouraging results. For the synthetic data we predict rather accurately the next move, and for the S&P500 we get some interesting results related to quadratic variation and volatility prediction.