SEE PROFILE

Abstract. This paper describes the “FAME ” multi-modal demonstrator, which integrates multiple communication modes – vision, speech and object manip-ulation – by combining the physical and virtual worlds to provide support for multi-cultural or multi-lingual communication and problem solving. The major challenges are automatic perception of human actions and understand-ing of dialogs between people from different cultural or linguistic backgrounds. The system acts as an information butler, which demonstrates context awareness using computer vision, speech and dialog modeling. The integrated computer-enhanced human-to-human communication has been publicly demonstrated at the FORUM2004 in Barcelona and at IST2004 in The Hague. Specifically, the “Interactive Space ” described features an “Augmented Table” for multi-cultural interaction, which allows several users at the same time to per-form multi-modal, cross-lingual document retrieval of audio-visual documents previously recorded by an “Intelligent Cameraman ” during a week-long seminar.