Robot Skill Learning Through Intelligent Experimentation

Thesis (Ph. D.)--University of Rochester. Dept. of Computer Science, 1995. Simultaneously published in the Technical Report series.In robot skill learning the robot must obtain data for training by executing expensive practice trials and recording their results. The thesis is that the high cost of acquiring training data is the limiting factor in the performance of skill learners. Since the data is obtained from practice trials, it is important that the system make intelligent choices about what actions to attempt while practicing. In this dissertation we present several algorithms for intelligent experimentation in skill learning. In open-loop skills the execution goal is presented and the controller must then choose all the control signals for the duration of the task. Learning is a high-dimensional search problem. The system must associate a sequence of actions with each commandable goal. We propose an algorithm that selects practice actions most likely to improve performance by making use of information gained on previous trials. On the problem of learning to throw a ball using a robot with a flexible link, the algorithm takes only 100 trials to find a ``whipping'' motion for long throws. Most closed loop learners improve their performance by gradient descent on a cost function. The main drawback of this method is convergence to non-optimal local minima. We introduce the concept of cooperation as a means of escaping these local minima. We assume the existence of several coaches that each improve some aspect of the controller's performance. Switching training between coaches can help the controller avoid locally minimal solutions. On the task of curve tracing with an inverted pendulum the cooperative algorithm learns to track faster than with a traditional method. In an integrated system with scarce sensor resources it is preferable to perform tasks without sensing. We observe that closed loop learning can function as an efficient search technique for open-loop control. Our system starts with closed loop learning. As it improves its ability to control the plant, it replaces sensor information with estimates. The result is a controller that tracks long segments of a reference curve open loop