Improving the search of learning classifier systems through interpretable feature clustering

Learning Classifier Systems (LCS) are a well-known machine learning method, producing sets of interpretable rules in order to solve a variety of problems. Despite this, a common issue that these systems run into is the creation of unhelpful rules, caused by having multiple features in the data representing similar areas of knowledge. While we can logically know that these rules will not be useful in conjunction with each other, this is much more difficult for the algorithm to innately know. This paper presents an exploration into using clustering algorithms for feature selection in LCS, selecting features that represent each major cluster of feature information. Combined with the innate power of LCS at finding nonlinear decision boundaries, these selected features can achieve results close to that of the full feature set while reducing the training time required to reach those results. The feature selection performed is highly interpretable, allowing for different features to be selected while maintaining the information spread in the feature subset. Learning Classifier Systems (LCS) [1] are an Evolutionary Computation method that learns a population of rules in order to perform tasks. One common application for LCS systems is classification, where each rule makes an individual mapping of the class from an input, and these heuristics are combined to form an overall class prediction.