Published In Towards the Application of Reinforcement Learning to Undirected Developmental Learning

We consider the problem of how a learn-ing agent in a continuous and dynamic world can autonomously learn about itself, its en-vironment, and how to perform simple ac-tions. In previous work we showed how an agent could learn an abstraction consisting of contingencies and distinctions. In this paper we propose a method whereby an agent using this abstraction can create its own reinforce-ment learning problems. The agent generates an internal signal that motivates it to move into states in which a contingency will hold. The agent then uses reinforcement learning to learn to move to those states effectively. It can then use the knowledge acquired through re-inforcement learning as part of simple actions. We evaluate this work using a simulated phys-ical agent that affects its world using two con-tinuous motor variables and experiences its world using a set of fifteen continuous and discrete sensor variables. We show that by using this method the learning agent can au-tonomously generate reinforcement learning problems allowing it to do simple tasks, and we compare its performance to that of a hand-created reinforcement learner using tile cod-ing. 1