A learning classifier system approach to relational reinforcement learning

This article describes a learning classifier system (LCS) approach to relational reinforcement learning (RRL). The system, Foxcs-2, is a derivative of Xcs that learns rules expressed as definite clauses over first-order logic. By adopting the LCS approach, Foxcs-2, unlike many RRL systems, is a general, model-free and “tabula rasa” system. The change in representation from bit-strings in Xcs to first-order logic in Foxcs-2 necessitates modifications, described within, to support matching, covering, mutation and several other functions. Evaluation on inductive logic programming (ILP) and RRL tasks shows that the performance of Foxcs-2 is comparable to other systems. Further evaluation on RRL tasks highlights a significant advantage of Foxcs-2’s rule language: in some environments it is able to represent policies that are genuinely scalable; that is, policies that are independent of the size of the environment