Optimal estimation via nonanticipative rate distortion function and applications to time-varying Gauss-Markov processes

In this paper, we develop finite-time horizon causal filters for general processes taking values in Polish spaces using the nonanticipative rate distortion function (NRDF). Subsequently, we apply the NRDF to design optimal filters for time-varying vector-valued Gauss-Markov processes, subject to a mean-squared error (MSE) distortion. Unlike the classical Kalman filter design, the developed filters based on the NRDF are characterized parametrically by a dynamic reverse-waterfilling optimization problem obtained via Karush-Kuhn-Tucker conditions. We develop algorithms that provide, in general, tight upper bounds to the optimal solution to the dynamic reverse-waterfilling optimization problem subject to a total and per-letter MSE distortion constraint. Under certain conditions, these algorithms produce the optimal solutions. Further, we establish a universal lower bound on the total and per-letter MSE of any estimator of a Gaussian random process. Our theoretical framework is demonstrated via simple examples.Peer reviewe