Limitation of Markov models and event-based learning and optimization

We first illustrate the possible limitations of the widely-used Markov model and then introduce the concepts of events, event-based policies and event-based optimization. Compared with the state-based policies, event-based policies may utilize the "future" information and therefore may perform better. In addition, the number of events may scale to the system size while the number of states grows exponentially. The event-based approach is particularly efficient for systems with special structural properties. The solutions to the event-based optimization can be developed with a sensitivity-based view, which is developed recently for the area of stochastic learning and optimization. ©2008 IEEE