Fusion and coordination for multimodal interactive information presentation

Users require more effective and efficient means of interaction with increasingly complex information and new interactive devices. This document summarizes the results of the international Dagstuhl Seminar on Coordination and Fusion in Multimodal Interaction that took place at Schloss Dagstuhl in Germany October 27 through November 2, 2001 1. We first outline a research roadmap in the near and long term. Next we describe requirements and an abstract architecture for this class of systems. We then detail requirements for semantic representations and languages necessary to enable these systems. Finally, we describe data, annotation methodologies and tools necessary to further advance the field. We conclude with a recommended action plan for forward progress in the community. 1. 0 ROADMAP Figure 1 illustrates the roadmap in the near term, from 2002-2005 for the creation of mobile, human-centered intelligent multimodal interfaces. Three “lanes ” in the road identify three areas of research and development, including empirical and data driven models of multimodality, advanced methods for multimodal communication and toolkits for multimodal systems. The end of the road maps indicate the outcome i