Joint Filtering and Factorization for Recovering Latent Structure from Noisy Speech Data

We propose a joint filtering and factorization algorithm to re-cover latent structure from noisy speech. We incorporate the minimum variance distortionless response (MVDR) formula-tion within the non-negative matrix factorization (NMF) frame-work to derive a single, unified cost function for both filtering and factorization. Minimizing this cost function jointly opti-mizes three quantities – a filter that removes noise, a basis ma-trix that captures latent structure in the data, and an activation matrix that captures how the elements in the basis matrix can be linearly combined to reconstruct input data. Results show that the proposed algorithm recovers the speech basis matrix from noisy speech significantly better than NMF alone or Wiener fil-tering followed by NMF. Furthermore, PESQ scores show that our algorithm is a viable choice for speech denoising. Index Terms: NMF, MVDR, denoising, filtering. 1