Deriving the optimal modality combination for mobile search in multimedia databases

The European MIAMM-project (Multidimensional Information Access using Multiple Modalities) develops new concepts and techniques for multimodal dialogs that allow for fast and natural mobile access to multimedia databases. In an experiment we investigated different modality combinations (haptic-tactile, non-speech audio and a redundant combination) for conveying information on the size of a data-set. The results show that users perform better in the condition with redundant non-speech audio and haptic-tactile feedback and also prefer this condition. The knowledge gained in this experiment will be extended in the course of the MIAMM-project to possibly cover other contexts, tasks and to do testing in realistic environments