Sensor space segmentation for mobile robot learning

Robot learning such as reinforcement learning gener-ally needs a well-defined state space in order to con-verge. However, to build such a state space is one of the main issues of the robot learning because of the inter-dependence between state and action spaces, which resembles to the well known “chicken and egg” problem. This paper proposes two methods of action-based state space construction for vision-based mobile robots. Basic ideas common to the two methods to cope with the inter-dependence are that we define a state as a cluster of of input vectors from which the robot can reach the goal state or the state already ob-tained by a sequence of one kind action primitive re-gardless of its length, and that this sequence is defined as one action. The first method clusters the input vec-tors as hyper ellipsoids so that the whole state space is segmented into a state transition map in terms of action from which the optimal action sequence is ob-tained. In order to obtain the such a map, we need a sufficient number of data, which means longer learn-ing time. To cope with this, we proposed the second method by which a robot learns purposive behavior within less learning time by incrementally segment-ing the sensor space based on the experiences of the robot. The incremental segmentation is performed by constructing local models in the state space, which is based on the function approximation of the sensor outputs to reduce the learning time and on the rein-forcement signal to emerge a purposive behavior. To show the validity of the methods, we apply them to a soccer robot which tries to shoot a ball into a goal. The simulation and real experiments are shown