Reinforcement Learning for Online Control and Optimisation

An intelligent controller has the ability to analyse an unknown situation and to respond to it accordingly. Approximate dynamic programming, or reinforcement learning as it is more commonly known, in the form of Adaptive Critic Designs (ACDS), falls into this category (56). ACDs offer an interesting alternative for adaptive control and optimisation of highly nonlinear industrial processes. In this chapter, the action dependent adaptive critic (ADAC) (47) is used and a suitable second-order training algorithm is presented to ensure fast convergence and stability. The performance of the training algorithm is first compared in simulation for the control of an inverted pendulum. The ADAC scheme is then applied to the control of an aluminium subtrate disk grinding process where the learning is based on actual industrial historical data. Results here indicate that the ADAC controller can control the unloading thickness variation of the process to achieve a 33% reduction in rejects