Broadband Networked Audio: Entering the Era of Multisensory Data Distribution

As high bandwidth networks have become widespread, Internet audio, once relegated to the second class status of AM radio, is now recognized as an effective means of au-dio distribution for both pre-recorded and live transmis-sions, including support for distributed, real-time inter-action. Recent advances, including transmission of sur-round sound (e.g. Dolby Digital AC-3), uncompressed multichannel 96kHz/24 bit streaming, and low-latency networked audio, have each inspired a range of novel technologies and applications, ushering in a new era of audio in which distance no longer dictates limitations for high quality musical experiences and interaction. In the non-networked world, audio tends to be expe-rienced in conjunction with other modalities, and thus, efforts to advance the state of the art must not neglect the challenges of integrating high-fidelity audio with rich video and vibro-sensory data. In this context, this pa-per explores some of the issues involved in developing a scalable architecture and suitable protocols for broadband multicast network distribution of multimodal data. These issues include coding for heterogeneous clients, both with respect to their capabilities and needs, balancing the con-flicting demands of optimal immersive quality, minimal latency and maximal reliability, and suppressing echo in such a manner that interaction is improved without im-pairing perception of the original signal. As music is one of the most demanding of applica-tions, the research described here relates to our goal of supporting the activities of both performer and audience, or student and teacher, in the process of creating and ex-periencing music. Achieving success in these domains requires contributions ranging from psychology, signal processing, to network engineering. 1