Predicting S&P500 Monthly Direction with Informed Machine Learning

Abstract

We propose a systematic framework based on a dynamic functional causal graph in order to capture complexity and uncertainty on the financial markets, and then to predict the monthly direction of the S&P500 index. Our results highlight the relevance of (i) using the hierarchical causal graph model instead of modelling directly the S&P500 with its causal drivers (ii) taking into account different types of contexts (short and medium term) through latent variables (iii) using unstructured forward looking data from the Beige Book. The small size of our training data is compensated by the a priori knowledge on financial market. We obtain accuracy and F1-score of 70.9% and 67% compared to 64.1% and 50% for the industry benchmark on a period of over 25 years. By introducing a hierarchical interaction between drivers through a latent context variable, we improve performance of two recent works on same inputs.