Research on Modeling the Nonhomogenous Markov Decision Systems with Dynamic Bayesian Network

For the purpose of the further wide application of dynamic Bayesian networks (DBNs) to many real complex systems, a new approach was presented to improve the modeling of the Non time homogenous Markov Decision systems with DBNs, in which the extended hidden variables were introduced into the evolutional process to build Markov models required by the hypothesis conditions, a structure learning algorithm of DBNs was given from the incomplete data set and when the extended hidden variables are existed. The sufficient statistics of the subsequent time slices were estimated using Bayesian probability statistical method, and then the time-variant transition probabilities were learned using both of current sufficient statistics and estimated sufficient statistics. The theoretical analysis and simulation results show that the proposed approach is valid. Key words: Dynamic Bayesian network, Markov model, hidde