Event-based optimization for POMDPs and its application in portfolio management

Partially observable Markov decision processes(POMDPs) provide a framework for the optimization of Markov systems when there exist multiple sources of uncertainty: besides the system's stochastic dynamics, there are also observation noises. While POMDPs have many real applications, existing approaches to searching global optimal policy is computationally intractable even for systems with small sizes. In this paper, we apply the idea of the recently developed event-based optimization approach to study POMDP problems with infinite horizon setting. Based on this approach, a perturbation analysis based algorithm can be proposed to search for a local optimal policy. Further more, under a certain condition, a policy iteration type algorithm can be developed. We find that such a condition is satisfied for some partially observable systems in the financial engineering area. As an example, we discuss a portfolio management problem at the end of the paper. © 2011 IFAC