Training recurrent neurocontrollers for robustness with derivative-free Kalman filter

Abstract—We are interested in training neurocontrollers for robustness on discrete-time models of physical systems. Our neurocontrollers are implemented as recurrent neural networks (RNNs). A model of the system to be controlled is known to the ex-tent of parameters and/or signal uncertainties. Parameter values are drawn from a known distribution. For each instance of the model with specified parameters, a recurrent neurocontroller is trained by evaluating sensitivities of the model outputs to pertur-bations of the neurocontroller weights and incrementally updating the weights. Our training process strives to minimize a quadratic cost function averaged over many different models. In the end, the process yields a robust recurrent neurocontroller, which is ready for deployment with fixed weights. We employ a derivative-free Kalman filter algorithm proposed by Norgaard et al. and extended by Feldkamp et al. (2001) and Feldkamp et al. (2002) to neural network training. Our training algorithm combines effectiveness of a second-order training method with universal applicability to both differentiable and nondifferentiable systems. Our approach is that of model reference control, and it extends significantly the capabilities proposed by Prokhorov et al. (2001). We illustrate it with two examples. Index Terms—Derivative-free Kalman filter, neurocontroller, training for robustness, recurrent neural network (RNN)