Designing Conversational Interfaces With Multimodal Interaction

Our current research focuses on developing conversational interfaces to on-line applications through speech recognition technology. We have developed a prototype system that combines pen and speech input from the on-line user in a web-browser. VoiceLog is a voice-enabled connection to a web-server that allows one to obtain vehicle diagrams and to place orders for specific parts in these diagrams. VoiceLog features a novel client-server approach to speech recognition, modular reusable components and a simple Java-based interface. This paper briefly describes the system and its architecture including the handling of simultaneous input from pen and speech, the production of audio and visual feedback, and the management of multimodal dialogue. 1. INTRODUCTION Traditional keyboard and mouse interfaces are impractical on small portable devices. Spoken language systems, such as voice enabled browsing, offer an intuitive way of accessing the growing amount of on-line information. The next ge..