Study on spatial coherence of moving audio-visual objects

In media technology video and audio coding are often applied together. The quality of experience (QoE) in such audio-visual systems is influenced by several factors. Besides the dominant factors pure audio and video quality, the temporal coherence of the auditory and visual events plays a role. For common two-dimensional applications these factors are well investigated, understood and most data formats in coding and reproduction system are optimized to meet the perceptual requirements. More sophisticated media technology systems enable the reproduction of 3D audio and video. In such systems the spatial audiovisual coherence is of significant interest. While spatial coherence is well understood when audio and video are on fixed positions, hardly no studies exist for moving objects. In this paper perceptual tests are presented trying to investigate the perceptual threshold for moving audio-visual objects in the horizontal plane. An offset of less than 2° was detectable for trained expert listeners. For these the result is independent from the sound stimulus used (speech, pink noise) and from the offset direction (audio in front of video versus video in front of audio). However a small difference of the threshold dependent on the direction of movement (left-to-right versus right-to-left) was observed