Non-negative Sparse Representations for Speech Enhancement and Recognition

Speech recordings taken from real-world environments often contain background noises which degrade the speech signal and reduce its intelligibility. Such degradations also adversely affect the performance of digital systems that operate on the recorded speech such as mobile communication, automatic voice assistance, automatic speech recognition (ASR), hearing aids and speaker identification systems. Robustness of such systems to the various acoustic background noises is still a challenging problem despite decades of research and myriad of different approaches. Speech enhancement schemes aim at recovering the original speech signal by suppressing the noises to improve the speech intelligibility and are popularly used as a front-end for most of the applications. This dissertation concentrates on speech enhancement schemes for single-channel recordings of noisy speech. Traditional single-channel speech enhancement schemes such as Wiener filtering do not work satisfactorily well in the presence of background noises that vary over time. Alternatively, composite models that approximate the noisy speech as a linear combination of long-context atoms that model the spectro-temporal behaviour of speech and noise signals can be effectively used for suppressing such non-stationary noises. The main goals of this thesis are to develop novel composite models for a better speech enhancement quality and to investigate the application of these settings as a front-end for the various state-of-the-art ASR systems. In particular, composite models derived from the family of non-negative matrix factorisation (NMF) algorithms, that have been successfully used for separating individual signals from a noisy mixture, are proposed. This thesis describes a set of new algorithms based on this theory that can be broadly subdivided into three main (overlapping) sections. First, we propose a coupled dictionary based approach to the family of NMF-based speech enhancement systems. Typical NMF-based systems use the decomposition in lower dimensional spectro-temporal feature representations. Such feature spaces are preferred over the full-resolution frequency domain for their reduced computational complexity and the ability to better generalise to unseen noise cases. But the resulting noise suppression may be sub-optimal because a low-rank approximation is used to map the estimated speech and noise features to the full-resolution frequency domain. The proposed approach provides an efficient way to directly compute the full-resolution frequency estimates of speech and noise using coupled dictionaries: an input dictionary containing atoms from the desired feature space to obtain the decomposition and a coupled output dictionary composed of atoms from the full-resolution frequency domain. We also introduce the perceptually motivated modulation spectrogram features for the NMF-based tasks. The idea of using coupled dictionaries is then extended to define hybrid exemplar-spaces that are obtained by concatenating different spectro-temporal representations for a better speech and noise separation. The proposed systems were evaluated for various choices of input representations and yielded improved speech enhancement performances on the AURORA-2 and AURORA-4 databases. We further show that the proposed approaches result in improved speech recognition accuracies on the AURORA-2, AURORA-4 and the CHiME-3 challenge databases. Next, we propose a novel approach to address the difficult problem of single-channel speech enhancement under noisy and reverberant environments. Such recordings are comprised of the original speech, its reflections from various surfaces and the background noise. Thus, the speech enhancement schemes for such scenarios should be able to remove these reflections as well, apart from separating background noise from the target speech. The effect of such reflections are mathematically modelled as a convolution of the original speech with a room impulse response (RIR) that typically has a decaying nature over time. We propose a novel approximation of the noisy reverberant speech in the frequency domain and non-negative matrix deconvolution (NMD). In the proposed model, the RIR in the frequency domain is defined such that its decaying structure can also be estimated from the recording itself. The proposed model is evaluated on a synthetic dataset created by convolving the recordings from the TIMIT database with RIRs measured from different rooms and varying speaker-and-microphone locations, and adding background noises taken from the CHiME-1 corpus. Simulation results show that the proposed model results in a better RIR estimate over the existing model and improves various instrumental speech quality measures. Finally, we present an application of one of the proposed speech enhancement schemes together with an ASR setting in the field of clinical neuroscience for the pre-operative planning on patients with brain tumor. During the pre-operative planning, a neurosurgeon has to decide if the affected brain region is essential for the major functions such as motor movement and language related processes. To identify the functional relevance of a brain region for language related processes, a picture naming task together with magnetic stimulation of the relevant brain region has been used. The magnetic stimulation equipment produces impulsive noises which are also captured by the microphone. Currently, the accuracy and the reaction times of the responses are found manually from the recordings which is prone to substantial intra- and inter-observer variabilities especially in the presence of the noise from the equipment. A novel automatic and objective evaluation routine for the picture naming task using ASR and the proposed speech enhancement schemes is developed and is tested against the manual annotations on responses collected from 8 subjects.Baby D., ''Non-negative sparse representations for speech enhancement and recognition'', Proefschrift voorgedragen tot het behalen van het doctoraat in de ingenieurswetenschappen, KU Leuven, November 2016, Leuven, Belgium.nrpages: 156status: publishe