Context Aware Voice User Interfaces for Workflow Support

Audio is a significant factor in the design of the human computer interface in ubiquitous computing. The characteristics of the medium allow for a hands-free interaction without the need to switch the focus to a display (eyes-free).Moreover, determining the graphical capabilities of the wearable device to adapt the output to the device is not needed. But audio based interfaces are also challenging, since humans are visually oriented. The ubiquitous computing community recognized the advantages of the audio channel, but the restrictions inherent to the medium are mostly ignored. Authors of such systems who know about these challenges often look for a solution by using additional modalities, preferably visually oriented. This thesis analyses these challenges with respect to the human’s cognitive capabilities and shows a possible solution using audio design patterns. Users on the move face additional challenges, since wearable devices do not have the computational power and storage capacity of desktop PCs which are required to use speech recognizers and speech synthesizers off the shelf. Thus recognition and synthesis software running on such a device does not have the capabilities of their desktop size counterparts. Streaming technologies can use high-end voice technology on a server at the cost of network dependency. This thesis introduces a new approach combining both worlds. In order to aid the user best, context information is used, location at first place. The context information is obtained from the current environment and from the task that she wants to accomplish. workflow engines are a standard for storing and controlling tasks as processes. Additionally, workflow engines consider the data flow which is a fundamental aspect of each process. Initial approaches to using context information in workflow engines exist but they lack a reusable concept.This thesis introduces such a concept. The process descriptions in workflow engines are modality independent in order to have reusable business logic. Modality dependant specifications of the current activity reside outside the workflow engine and are stored in a meta format that can be used to address multiple modalities. The user interfaces are generated by code generators at run-time. Current concepts consider neither a separation of modality independent business logic nor do they regard data flow. If a user is working on multiple processes in parallel, she needs support in returning into the context of an interrupted process. The challenges inherent to audio are not well suited to aid her resuming the task efficiently. In this thesis, we introduce a first concept of an audio-only support for task resumption