Reinforcement learning for spatial processes

Abstract: Markov decision processes (MPDs) have become a popular model for real-world problems of planning under uncertainty. A vide range of applications has been published within the fields of natural resources management, forestry, agricultural economics, and robotics. An MDP can be used to represent and optimize the management of an environment. A state variable is used to represent the current state of the environment and the interaction with the environment is expressed with an action variable, allowing for stochastic transition from one state to another state. A transition function is used to express the transition between the different states of the environment, and a reward function expresses the rewards received by applying an action when the environment is in a specific state. MDPs can thereby be used to optimize management strategies for problems with uncertain effects of actions, multiple and conflicting objectives, stochastic events, and stochastic environments. A common problem when optimizing management strategies for natural resources is that large areas have to be considered, rendering the state and action space of the MDP large. For example, an agricultural area consisting of thousands of crop fields, or forests consisting of thousands of stands. Also, spatial aspects may have to be considered. Some examples are management of epidemics, invasive or endangered species, management to prevent wind damage and wild fire. However, the structure of the problem can be used t