Eyes-Free Environmental Awareness for Navigation

El-Shimy D, Grond F, Olmos A, Cooperstock JR. Eyes-Free Environmental Awareness for Navigation. Journal on Multimodal User Interfaces. 2012;5(3-4):131-141.We consider the challenge of delivering location-based information through rich audio representations of the environment, and the associated opportunities that such an approach offers to support navigation tasks. This challenge is addressed by In-Situ Audio Services, or ISAS, a system intended primarily for use by the blind and visually impaired communities. It employs spatialized audio rendering to convey the relevant content, which may include information about the immediate surroundings, such as restaurants, cultural sites, public transportation locations, and other points of interest. Information is aggregated mostly from online data resources, converted using text-to-speech technology, and ``displayed'', either as speech or more abstract audio icons, through a location-aware mobile device or smartphone. This is suitable not only for the specific constraints of the target population, but is equally useful for general mobile users whose visual attention is otherwise occupied with navigation. We designed and conducted an experiment to evaluate various techniques for delivering spatialized audio content to users via interactive auditory maps. The results of this experiment and its implications to our project are described here