Data Visualization Methods for the Blind using Force Feedback and Sonification

Research in the field of scientific visualization has begun to articulate systematic methods for mapping data to a perceivable form. Most work in this area has focused on the visual sense. For blind people in particular, systematic visualization methods which utilize other sense need further development. In this work we develop methods for adding aural feedback to an existing haptic force feedback interface to create a multimodal visualization system. We examine some fundamental components of a visualization system which include the following: characterization of the data, definition of user directives and interpretation aims, cataloging of sensual representations of information, and finally, matching the data and user interpretation aims with the appropriate sensual representations. We focus on the last two components as they relate to the aural and haptic sensor. Cataloging of sensual representations is drawn form current research in sonification and haptics. The matching procedure can be thought of as a type of encoding which should be the inverse of the decoding mechanism of our aural and haptic systems. Topics in human perception are discussed, and issues related to the interaction between the two sensor are addressed. We have implemented a visualization system in the from of a Windows NT application using a sound card and a 3 DOF point interaction haptic interface. The system present a 2D data set primarily as a polygonal haptic surface although other capabilities of the haptic sensor are utilized such as texture discernment. In addition, real time aural feedback is presented as the user explores the surface. Parameters of sound such as pitch and spectral content are used to convey information. Evaluation of the system's effectiveness as an assistive technology for the blind reveals that combining aural and haptic feedback improves visualization over using either of the two senses alone