Reinforcement Learning of Parameters for Humanoid Rhythmic Walking based on Visual Information

This paper presents a method for learning the parameters of rhythmic walking to generate a purposive motion. The controller consists of the two layers. Rhythmic walking is realized by the lower layer controller which adjusts the speed of the phase on the desired trajectory de-pending on the sensor information. The upper layer controller learns (1) the feasible parame-ter sets that enable a stable walking for a robot, (2) the causal relationship between the walk-ing parameters to be given to the lower layer controller and the change of the sensor infor-mation, and (3) the feasible rhythmic walking parameters by reinforcement learning so that a robot can reach to the goal based on the vi-sual information. The method was examined in the real robot, and it learns to reach the ball and to shoot it into the goal in the context of RoboCupSoccer competition.