Prediction of stock market returns and direction : application of machine learning models

Abstract: Prediction of market direction has gained more attention than the prediction of point returns over the past few years. Market direction prediction is essential in determining buy and sell investment strategies. A correct forecast of the market trend will lead investors to make knowledgeable decisions about their future investments. This study seeks to predict the market direction of two developed markets (USA and UK) and two emerging markets (SA and Brazil) using five machine learning techniques, namely support vector machines (SVM), decision trees (DTs), random forest (RF), k-nearest neighbours (K-NN) and linear discriminant analysis (LDA), which is considered as the benchmark model. The choice of the four stock markets is due to developed and emerging markets suffering from structural breaks in different ways. Emerging markets are more volatile and are prone to suffer from more structural breaks than developed markets. Therefore, the aim of this study is to compare the performance and ability of the above-mentioned machine learning techniques to predict in both an environment with more structural breaks and one with less structural breaks. The main objective is hence to find out which one of these techniques performs better in consistently predicting stock market directions. The techniques’ performances are measured and evaluated using matrices such as the confusion matrix, receiver operating characteristic (ROC), accuracy tests, precision, balanced accuracy and F1 score. To avoid the effect of bias, we split the entire data set into 10 folds and use the crossvalidation method with bootstrapping technique. Using daily stock prices of the S&P 500, FTSE 100, All Share Index (ALSI) and Bovespa, a representation of the respective stock market indices of the abovementioned countries was obtained, as well as their respective price earnings (PE) ratios and dividend yields from 03 July 1995 to 28 August 2018. Our empirical results show that the S&P 500, FTSE 100 and ALSI market directions are primarily driven by their previous day stock returns and dividend yields. The Bovespa market direction, on the other hand, is more driven by its previous day return as well as its PE ratio. Using the confusion matrix, we find that the RF and DTs are the best models in predicting the market direction of S&P 500, FTSE 100 and ALSI, followed by K-NN, SVM and LDA. K-NN and DT failed to predict the upward trend of the Brazilian stock market index.M.Com. (Financial Economics