Using decision trees as the answer network in temporal-difference networks

Temporal difference networks (or TD-Nets) offer a framework for predictive state representations. TD-Nets break up into two parts: the question network and the answer network. The question network defines which questions about future observations are of importance, while the answer network provides a way to update the answers to those questions as the environment changes. Currently, TD-Nets use logistic regression functions to represent the answer networks. We propose the use of probability trees in their stead. Trees offer a different but powerful way of generalisation and using them may be beneficial in a number of applications. Moreover, we believe this aids in a better understanding of the strengths and weaknesses of TD-Nets and represents an important first step towards the application of temporal difference networks in environments with more extensive, i.e. complex and numerous, observations than those currently employed. We compare the learning behavior of TD-Nets using logistic regression and probability trees using an array of experiments in two simple grid worlds and a ring world.acceptance rate = 23%status: publishe