A Retrospective on Adaptive Dynamic Programming for Control

Some three decades ago, certain computational intelligence methods of reinforcement learning were recognized as implementing an approximation of Bellman\u27s Dynamic Programming method, which is known in the controls community as an important tool for designing optimal control policies for nonlinear plants and sequential decision making. Significant theoretical and practical developments have occurred within this arena, mostly in the past decade, with the methodology now usually referred to as Adaptive Dynamic Programming (ADP). The objective of this paper is to provide a retrospective of selected threads of such developments. In addition, a commentary is offered concerning present status of ADP, and threads for future research and development within the controls field are suggested. © 2009 IEEE