Accelerating Critic Learning in Approximate Dynamic Programming via Value Templates and Perceptual Learning

The concept of value templates and perceptual learning are introduced as refinements to the reinforcement learning (RL) paradigm. We demonstrate a method for accelerating Dual Heuristic Programming (DHP) critic training using value templates and perceptual learning. Both faster and more stable learning are achieved by using the value template and utilizing its inherent constraints to regularize the perceptual learning task. The method is demonstrated by tuning a neurofuzzy control system for a highly nonlinear 2nd order plant proposed by Sanner and Slotine. We take advantage of the TSK model framework throughout to keep the controller, critic, and model components used in DHP highly interpretable