Towards Ambient Intelligence: Multimodal Computers that Understand Our Intentions

This paper discusses the state-of-the-art and future developments of conversational multimodal interaction with computers, one of the corner stones of all future eCommerce and eWork applications. On the basis of past and present experience it is argued that natural, conversational interaction can only be achieved on the basis of powerful cognitive models of multimodal human-machine interaction. The paper explains how these models can be developed, how they can be used to make the design of multimodal applications more cost-effective, and how they facilitate non-experts in using complex applications. In addition the paper also identifies a number of threats that must be recognized by system and service developers. Throughout the paper we use a prototype system for the design of bathrooms as an example to illustrate the major points. We also discuss how the principles of founding multimodal interaction on sound cognitive models can be generalized to different application domains