Learning Locomotion Controllers with a Policy Iteration Algorithm

Abstract—Reinforcement learning and policy search methods can in principle solve a wide range of control tasks automatically. However, practical robotic applications of policy search typically require a carefully designed policy representation that is specific to each task [2]. For high dimensional robotics tasks where value function estimation is impractical, policy gradient methods usually achieve the best results [6]. However, these methods assume that the policy return is a smooth function of the parameters. Since locomotion is inherently a hybrid task that combines both smooth and discontinuous dynamics [3], such methods are difficult to apply to locomotion without a carefully engineered policy parameterization that subsumes the nonsmooth aspects of the problem. We present a policy optimization method that can effectively handle discontinuous dynamics and learn locomotion controller