Multimodal user interface for internet

This paper introduces new conceptions that include network multimodal user interface model, context-based retrieving based on the integration of speech and mouse-pointing, integration strategy based on the principal modality, natural language understanding applying to multimodal interaction. Two instances of the project are given: one is a Chinese-voice-driven multimodal file manager, and the other is a multimodal ActiveX control for Internet-shopping. The experiment results demonstrate the techniques above mentioned conduce to the improvement of naturalness and efficiency of HCI.EI0121270-12752