Predicting S&P500 Index direction with Transfer Learning and a Causal Graph as main Input
This work provides a method for investors and financial analysts to predict S&P500 index movements, offering strong specific gains over existing benchmarks.
This paper proposes a multi-tasking framework to predict the monthly direction of the S&P500 index. It achieves an Accuracy of 74.3%, F1-score of 67%, and Matthew Correlation of 0.42 over a 12-year test period, outperforming industry benchmarks.
We propose a unified multi-tasking framework to represent the complex and uncertain causal process of financial market dynamics, and then to predict the movement of any type of index with an application on the monthly direction of the S&P500 index. our solution is based on three main pillars: (i) the use of transfer learning to share knowledge and feature (representation, learning) between all financial markets, increase the size of the training sample and preserve the stability between training, validation and test sample. (ii) The combination of multidisciplinary knowledge (Financial economics, behavioral finance, market microstructure and portfolio construction theories) to represent a global top-down dynamics of any financial market, through a graph. (iii) The integration of forward looking unstructured data, different types of contexts (long, medium and short term) through latent variables/nodes and then, use a unique VAE network (parameter sharing) to learn simultaneously their distributional representation. We obtain Accuracy, F1-score, and Matthew Correlation of 74.3 %, 67 % and 0.42 above the industry and other benchmark on 12 years test period which include three unstable and difficult sub-period to predict.