Published In Continuous-Domain Reinforcement Learning Using a Learned Qualitative State Representation∗

We present a method that allows an agent to learn a qualitative state representation that can be applied to re-inforcement learning. By exploring the environment the agent is able to learn an abstraction that consists of land-marks that break the space into qualitative regions, and rules that predict changes in qualitative state. For each predictive rule the agent learns a context consisting of qualitative variables that predicts when the rule will be successful. The regions of this context in with the rule is likely to succeed serve as a natural goals for reinforce-ment learning. The reinforcement learning problems created by the agent are simple because the learned ab-straction provides a mapping from the continuous input and motor variables to discrete states that aligns with the dynamics of the environment