Distributed control of situated assistance in large domains with many tasks

This paper tackles the problem of building situated prompt-ing and assistance systems for guiding a human with a cog-nitive disability through a large domain containing multiple tasks. This problem is challenging because the target pop-ulation has difficulty maintaining goals, recalling necessary steps and recognizing objects and potential actions (affor-dances), and therefore may not appear to be acting rationally. Prompts or cues from an automated system can be very help-ful in this regard, but the domain is inherently partially ob-servable due to sensor noise and uncertain human behaviours, making the task of selecting an appropriate prompt very chal-lenging. Prior work has shown how such automated assis-tance for a single task can be modeled as a partially observ-able Markov decision process (POMDP). In this paper, we generalise this to multiple tasks, and show how to build a scalable, distributed and hierarchical controller. We demon-strate the algorithm in a set of simulated domains and show it can perform as well as the full model in many cases, and can give solutions to large problems (over 1015 states and 109 observations) for which the full model fails to find a policy.