NSF-CAREER: The Listening Machine Annual Report 2006

In 2006, we continued our investigations in machine understanding of complex audio signals in the domains of environmental ‘personal audio ’ recordings, for music audio, and for marine mammal sounds. 1.1.1 Environmental Sound Separation The problem of isolating and identifying individual sources in environmental record-ings has been pursued from a number of directions. RonWeiss has continued work on separation sound mixtures based on the constraints provided by learned mod-els of source signal behavior (specifically speech); this work is now feeding into our new NSF supported collaboration on Separating Speech from Speech Noise (IIS-05-35168). We published a description of this work at ICASSP [5], as well as work on estimating time-frequency masks based on a trained “Relevance Vector Machine ” classifier which we presented at the SAPA workshop [15]. 1 Spatial information can be very helpful in separating sounds, particularly when multiple channels are available. We have been looking at recovering information on source location from two-channel recordings such as might be recorded from a ‘dummy head ’ microphone. Although this is a well-studied problem, the pres-ence of reverberation (where reflections from walls can appear as a large number of additional ‘ghost ’ sources) generally defeats current techniques. We used sta-tistical modeling to make a precise fit to the between-channel characteristics that arise from reverberation, enabling accurate inference of the position of multiple, overlapping sound sources, and resulting in a time-frequency mask to recover the signal for each source. This work was presented at the SAPA workshop [10] an