Learning to control a biped with feet

Abstract—This paper investigates the learning of a controller for a flat-footed bipedal robot using reinforcement learning to optimally respond to (1) external disturbances induced by stepping on objects or being pushed, and (2) rapid reversal of demanded walk direction. The reinforcement learning method employed learns an optimal policy by actuating the ankle joints to assert pressure along the support foot, and optionally the leg joints to determine the next swing foot placement. The controller is learnt in simulation using an inverted pendulum model and the policy transferred to two small physical humanoid robots. I