Optimal control of stochastic systems with costly observations - the general markovian model and the LQG problem

Abstract—In this paper, we examine a discrete-time stochas-tic control problem in which there are a number of observation options available to the controller, with varying associated costs. The observation costs are added to the running cost of the optimization criterion and the resulting optimal control problem is investigated. This problem is motivated by the wide deployment of networked control systems and data fusion. Since only part of the observation information is available at each time step, the controller has to balance the system performance with the penalty of the requested information (query). We first formulate the problem for a general partially observed Markov decision process (POMDP) model and then specialize to the stochastic LQG problem, where we show that the separation principle still holds. Moreover we show that the effect of the observation cost is manifested on the estimation strategy as follows: instead of a Kalman filter with gain determined by the algebraic Riccati equation, the optimal estimator includes, in addition, a query strategy which is characterized by a dynamic programming equation. The structure of the optimal query for a one-dimensional system is studied analytically and simulated with numerical examples. I