Telepresence by Spatial Audio-Visual Output

A new approach to telepresence is presented in which binaural audio and stereoscopic video are used to mediate the spatiality of distant remote environments. The physiology of human audio-visual perception is taken into account as well as aspects of capturing, transmission, and display of binaural audio and stereoscopic video data. How the Internet can be used as tranmission channel is a part of the investigation of this thesis. This thesis describes how captured spatial audio signals can be encoded to minimize data complexity and to recover lost packets of audio data which have to be transmitted over the Internet. Strategies to prevent from wrong packet orders and different network latency situations are added to reach high fidelity in spatial audio. A dummy-head was constructed that provides inner ear microphones that were plugged into synthetic replicas of the human ears to capture audio as natural as possible. Two video cameras are used to capture stereoscopic video that is encoded and transmitted to the remote site. Advantages of different audio and video encoding and decoding strategies are discussed. Different network protocols are examined which can be used to transmit data over the Internet. Depending on the requirements for a telepresence system, recommendations for audio and video capturing, data transmission, audio and video display, and other requirements are presented. Several tests with audio and video applications were made to investigate if existing applications can be integrated into a telepresence system. How the missing features can be implemented to reach an immersive telepresence system with high quality in audio and video is shown in the last part