State estimation and optimal control of stochastic hybrid systems: Theory and applications

The Stochastic Hybrid System (SHS) is a class of complex stochastic dynamical systems with interacting continuous and discrete dynamics. Many cyber-physical systems with continuous/physical parts interconnected with discrete/logical elements can be well modeled as the SHS. This paper develops a set of theories and algorithms for state estimation and optimal control of the SHS. Firstly, to describe the time-evolution of the SHS, two kinds of dynamical models are proposed: the Discrete-Time SHS (DTSHS) and the Continuous-Time SHS (CTSHS). Then, a hybrid estimation algorithm is developed to estimate both the continuous and discrete states of the DTSHS with noisy sensor measurements. Also, the proposed hybrid estimation algorithm is applied to aircraft surveillance sensor fusion, aircraft trajectory prediction and conflict detection for air traffic management. As for the CTSHS, its state estimation problem is mathematically formulated and solved using proposed filtering equations which are able to characterize the time evolution of the probability distribution of the hybrid state estimates. The filtering equations are numerically solved via a Markov Chain (MC) approximation algorithm. It is proved that this algorithm is able to give a result that weakly converges to the exact solution of the filtering equation when the resolution of the approximation converges to zero. Finally, the optimal control problem for the DTSHS is solved via a separable\u27\u27 control scheme: the controller uses the proposed hybrid estimation algorithm to estimate the state of the DTSHS and the optimal control inputs are computed from the state estimates. To numerically compute the optimal mapping from the state estimates to the admissable control inputs, the Q-learning approach is applied, which is based on discretizing the distribution function space of all possible state estimates. The proposed optimal control algorithm is validated through a spacecraft rendezvous example