Integrating Reinforcement Learning and Declarative Programming to Learn Causal Laws in Dynamic Domains

Abstract. Robots deployed to assist and collaborate with humans in complex domains need the ability to represent and reason with incomplete domain knowl-edge, and to learn from minimal feedback obtained from non-expert human par-ticipants. This paper presents an architecture that combines the complementary strengths of Reinforcement Learning (RL) and declarative programming to sup-port such commonsense reasoning and incremental learning of the rules govern-ing the domain dynamics. Answer Set Prolog (ASP), a declarative language, is used to represent domain knowledge. The robot’s current beliefs, obtained by in-ference in the ASP program, are used to formulate the task of learning previously unknown domain rules as an RL problem. The learned rules are, in turn, encoded in the ASP program and used to plan action sequences for subsequent tasks. The architecture is illustrated and evaluated in the context of a simulated robot that plans action sequences to arrange tabletop objects in desired configurations.