Machine Intentionality, the Moral Status of Machines, and the Composition Problem

ABSTRACT: According to the most popular theories of intentionality, a family of theories we will refer to as “functional intentionality, ” a machine can have genuine intentional states so long as it has functionally characterizable mental states that are causally hooked up to the world in the right way. This paper considers a detailed description of a robot that seems to meet the conditions of functional intentionality, but which falls victim to what I call “the composition problem. ” One obvious way to escape the problem (arguably, the only way) is if the robot can be shown to be a moral patient – to deserve a particular moral status. If so, it isn’t clear how functional intentionality could remain plausible (something like “phenomenal intentionality ” would be required). Finally, while it would have seemed that a reasonable strategy for establishing the moral status of intelligent machines would be to demonstrate that the machine possessed genuine intentionality, the composition argument suggests that the order of precedence is reversed: The machine must first be shown to possess a particular moral status before it is a candidate for having genuine intentionality. In this paper, an answer will be sought to the question: “What properties must a machine have if it is to have genuine beliefs about objects in the world?” It would seem to be an altogether different question – about the foundations of moral personhood rather than about intentionality – to ask: “What properties must a machine have if it is to be an object of moral concern (if I am to have any obligations with respect to it)?” Most believe that an answer to the first question does not engage issues of morality or moral agency. The common assumption is that we must first determine what intentionality is, and whether non-biological machines are capable of possessing it, before we ask what, if any, our moral obligations might be with respect to such systems. I will argue that the order of conceptual priority is reversed. The second question must be answered before th