Chapter 5 Document-centric and

multimodal meeting assistants⇤ In this chapter, we will show how human-computer interaction (HCI) can benefit to meeting support technology, by reviewing the evolution of HCI research from a smart meeting minutes application, with document-centric meeting browsers, towards more user-centric assistance tools for meetings. The chapter will exemplify these trends with research performed at the University of Fribourg, within the IM2 NCCR. The chapter first focuses on a smart meeting minutes application, which consists in recording meetings and analyzing them in order to automatically produce minutes. More specifically, we show the importance of analyzing documents that are discussed or shown during meetings. Multimodal docu-ment alignment techniques link documents with other types of media such as meeting transcripts, audio, and video recordings. This opens the possi-bility of developing document-centric meeting browsers that use documents as indexes towards meeting parts and to the associated audio-video records. For instance, clicking on a part of a document will play the audio-video sequences of the meeting in which this document part was discussed or pro-jected. Further on, additional explorations are discussed, including mainly ego-centric and cross-meeting browsing of large archives of multimedia meeting data through keywords, links and personal cues. A multimodal toolkit is presented, which facilitates the development of multimodal user interfaces (using voice, gestures, eyes movements, etc., to interact with machines) that can be used during meetings as online assistants to improve teamwork. For instance, a tabletop application was developed with this toolkit to facilitate brainstorming in groups. ⇤This chapter was written by Denis Lalanne