Multimodal meeting event recognition fusing three different types of recognition techiques

In this paper we introduce a new method for recognizing meeting events. In the present case the boundaries of the meeting segments are a priori known. The recognition task is performed using a classifier fusion technique that combines the three different used approaches for meeting event recognition. The results show that by classifier fusion a more stable result can be achieved than using only one classifier. For our experiments with meeting segmentation and meeting event recognition we make use of special scripted meetings that were recorded in the IDIAP Smart Meeting Room [1]. These recorded meetings consist of a set of predefined meeting events in a specific order. The appearing events were: Monologue1 to Monologue4 (one of the four participants (1 to 4) speaks continuously), discussion (all participants engage in a discussion), note-taking (all participants write notes), white-board (one participant at front of room, makes notes on the white board), presentation (one participant at front of room makes a presentation), consensus (all participants express consensus) and disagreement (all participants express disagreement). Thus we differentiate 10 different events. The classification task is performed by three different approaches, that are combined via Late Semantic Fusion: A static approach using simulated results of specialized recognizers, a dynamic approach using the audio files, and a dynamic approach using the tran-scriptions. The basic idea of the first approach is to take advantage of the results of various specialized recognizers, like gesture recognizers, person trackers and so on. As long as these recognizers only exist at a developing level we have to simulate them. This has been done for all scripted meetings by annotating the various actions that each person is performing e.g. talking, writing, pointing, standing etc. For each period of time global statistics are calculated that tell the percentage of an action in that period. Carefully selected items are then put together into a feature vector. The classification of an unknown feature vector is performed by a multi-clas