Reinforcement Learning Transfer using a Sparse Coded Inter-Task Mapping

Abstract. Reinforcement learning agents can successfully learn in a variety of difficult tasks. A fundamental problem is that they learn slowly in complex envi-ronments, inspiring the development of speedup methods such as transfer learn-ing. Transfer improves learning by reusing learned behaviors in similar tasks, usu-ally via an inter-task mapping, which defines how a pair of tasks are related. This paper proposes a novel transfer learning technique to autonomously construct an inter-task mapping by using a novel combinations of sparse coding, sparse pro-jection learning, and sparse pseudo-inputs gaussian processes. Experiments show successful transfer of information between two very different domains: the moun-tain car and the pole swing-up task. This paper empirically shows that the learned inter-task mapping can be successfully used to (1) improve the performance of a learned policy on a fixed number of samples, (2) reduce the learning times needed by the algorithms to converge to a policy on a fixed number of samples, and (3) converge faster to a near-optimal policy given a large amount of samples.