Efficient value iteration using partitioned models

Abstract — In order to solve large-scale value iteration problems, more intelligent allocation of computing time is needed. We introduce the idea of an information frontier, which allows us to identify maximally productive regions of the problem space. We present a potential information flow metric which allows us to quantify the frontier precisely. We also introduce a partitioning scheme, which effectively combines with the flow metric to reduce the complexity of problematic operations. The framework is powerful, and can be used to parallelize valueiteration, effectively manage memory in large-scale problems, or further multi-agent cooperative solution methodologies. A complete algorithm is developed and successfully tested on several problems. Experimental evidence is presented which demonstrates the efficacy of the approach. I