New Algorithms for Evolving Robust Genetic Programming Solutions in Dynamic Environments with a Real World Case Study in Hedge Fund Stock Selection

This thesis presents three new genetic programming (GP) algorithms designed to enhance robustness of solutions evolved in highly dynamic environments and investigates the application of the new algorithms to financial time series analysis. The research is motivated by the following thesis question: what are viable strategies to enhance the robustness of GP individuals when the environment of a task being optimized or learned by a GP system is characterized by large, rapid, frequent and low-predictability changes? The vast majority of existing techniques aim to track dynamics of optima in very simple dynamic environments. But the research area in improving robustness in dynamic environments characterized by large, frequent and unpredictable changes is not yet widely explored. The three new algorithms were designed specifically to evolve robust solutions in these environments. The first algorithm ‘behavioural diversity preservation’ is a novel diversity preservation technique. The algorithm evolves more robust solutions by preserving population phenotypic diversity through the reduction of their behavioural intercorrelation and the promotion of individuals with unique behaviour. The second algorithm ‘multiple-scenario training’ is a novel population training and evaluation technique. The algorithm evolves more robust solutions by training a population simultaneously across a set of pre-constructed environment scenarios and by using a ‘consistency-adjusted’ fitness measure to favour individuals performing well across the entire range of environment scenarios. The third algorithm ‘committee voting’ is a novel ‘final solution’ selection technique. The algorithm enhances robustness by breaking away from ‘best-of-run’ tradition, creating a solution based on a majority-voting committee structure consisting of individuals evolved in a range of diverse environmental dynamics. The thesis introduces a comprehensive real-world case application for the evaluation experiments. The case is a hedge fund stock selection application for a typical long-short marketneutral equity strategy in the Malaysian stock market. The underlying technology of the stock selection system is GP which assists to select stocks by exploiting the underlying nonlinear relationship between diverse ranges of influencing factors. The three proposed algorithms are all applied to this case study during evaluation. The results of experiments based on the case study demonstrate that all three new algo-rithms overwhelmingly outperform canonical GP in two aspects of the robustness criteria and conclude they are viable strategies for improving robustness of GP individuals when the environment of a task being optimized or learned by a GP system is characterized by large, sudden, frequent and unpredictable changes