INCREMENTAL SYNTHESIS OF OPTIMAL CONTROL LAWS USING LEARNING ALGORITHMS

Learning systems represent an approach to optimal control law design for situations where initial model uncertainty precludes the use of robust, fixed control laws. This thesis analyzes a variety of techniques for the incremental synthesis of optimal control laws, where the descriptor incremental implies that an on-line implementation filters the information acquired through real-time interactions with the plant and the operating environment. A direct/indirect framework is proposed as a means of classifying approaches to learning optimal control laws. Within this framework, relationships among existing direct algorithms are examined, and a specific class of indirect control laws is developed. Direct learning control implies that the feedback loop that motivates the learn-ing process is closed around system performance. Reinforcement learning is a type of direct learning technique with origins in the prediction of animal learning phenom-ena that is largely restricted to discrete input and output spaces. Three algorithms that employ the concept of reinforcement learning are presented: the Associativ