Approximating Spatial Markov Decision Processes For Environmental Management

Abstract: In this paper we will focus on spatialized decision problems which we propose to model in the framework of (highly) multidimensional Markov Decision Processes (MDPs) which exhibit only local dependencies between variables. We propose to approximate a Markov chain on a multidimensional random variable by a Markov chain on a set of weakly dependent random variables. This allows to (approximately) solve multidimensional MDPs with hundreds of variables, to the price of a loss of exactness of the process model. The method is mostly empirical yet, however it allows to deal with decision problems far larger than the one usually dealt with in the MDP framework