Interactive visualization and sonification for monitoring complex processes

This paper introduces AVDisplay, a versatile auditory and visual display for monitoring, querying and accessing information about modules or processes in complex systems. In the context of a collaborative research effort (SFB360, artificial communicators) at Bielefeld University, a cognitive robotics system for human-machine interaction is being developed. The AVDisplay provides the central interface for monitoring and debugging this system, currently involving about 20 computers hosting more than 30 com-plex processes. The display is designed to provide a summary over the system’s activities combining visualization and sonifica-tion techniques. The dynamic visualization allows inference of correlated activity of processes. A habituation simulation process automatically sets a perceptional focus on interesting and relevant process activities. The sonification part is designed to integrate emotional aspects – if the system suffers from poor sensory qual-ity, the sound conveys this by sounding uncomfortable. 1