One hundred challenge problems for logical formalizations of commonsense psychology

We present a new set of challenge problems for the logical formalization of commonsense knowledge, called Triangle-COPA. This set of one hundred problems is smaller than other recent commonsense reasoning question sets, but is unique in that it is specifically designed to support the development of logic-based commonsense theories, via two means. First, questions and potential answers are encoded in logical form using a fixed vocabulary of predicates, eliminating the need for sophisticated natural language processing pipelines. Second, the domain of the questions is tightly constrained so as to focus formalization efforts on one area of inference, namely the commonsense reasoning that people do about human psychology. We describe the authoring methodology used to create this problem set, and our analysis of the scope of requisite commonsense knowledge. We then show an example of how problems can be solved using an implementation of weighted abduction