From non-deterministic to probabilistic planning with the help of statistical relational learning

Using machine learning techniques for planning is getting increasingly more important in recent years. Various aspects of action models can be induced from data and then exploited for planning. For probabilistic planning, natural candidates are learning of action effects and their probabilities. For expressive formalisms such as PPDDL, this is a difficult prob- lem since they can introduce easily a hidden data problem; the fact that multiple action outcomes may have generated the same experienced state transitions in the data. Furthermore the action effects might be factored such that this prob- lem requires solving a constraint satisfaction problem within an expectation maximization scheme. In this paper we outline how to utilize recent techniques from the field of statistical relational learning for this problem. More specifically, we show how techniques developed for the CPT-L model of relational probabilistic sequences can be applied to the problem of learning probabilities in a PPDDL model. A CPT-L model concisely specify a Markov chain over arbitrary numbers of objects in the domain and simultaneous applications of multiple actions. The use of efficient BDD-style representations allows for fast and efficient learning in such domains. Even efficient online learning is possible as we will show in this paper. We relate to other learning approaches for similar domains and highlight the opportunities for incorporating our approach into architectures that can plan, execute the plan, and learn from the outcomes, in an online and incremental fashion.status: publishe