Do you get it? User-evaluated explainable BDI agents

Abstract. In this paper we focus on explaining to humans the behavior of au-tonomous agents, i.e., explainable agents. Explainable agents are useful for many reasons including scenario-based training (e.g. disaster training), tutor and peda-gogical systems, agent development and debugging, gaming, and interactive sto-rytelling. As the aim is to generate for humans plausible and insightful expla-nations, user evaluation of different explanations is essential. In this paper we test the hypothesis that different explanation types are needed to explain different types of actions. We present three different, generically applicable, algorithms that automatically generate different types of explanations for actions of BDI-based agents. Quantitative analysis of a user experiment (n=30), in which users rated the usefulness and naturalness of each explanation type for different agent actions, supports our hypothesis. In addition, we present feedback from the users about how they would explain the actions themselves. Finally, we hypothesize guidelines relevant for the development of explainable BDI agents1.