Efcient Model Learning for Dialog Management

Intelligent planning algorithms such as the Partially Observable Markov Decision Process (POMDP) have succeeded in dialog man-agement applications [10, 11, 12] because they are robust to the in-herent uncertainty of human interaction. Like all dialog planning systems, however, POMDPs require an accurate model of the user (e.g., what the user might say or want). POMDPs are generally specied using a large probabilistic model with many parameters. These parameters are difcult to specify from domain knowledge, and gathering enough data to estimate the parameters accurately a priori is expensive. In this paper, we take a Bayesian approach to learning the user model simultaneously with dialog manager policy. At the heart of our approach is an efcient incremental update algorithm that al-lows the dialog manager to replan just long enough to improve the current dialog policy given data from recent interactions. The up-date process has a relatively small computational cost, preventing long delays in the interaction. We are able to demonstrate a robust dialog manager that learns from interaction data, out-performing a hand-coded model in simulation and in a robotic wheelchair appli-cation