Wavelet Multiresolution Analysis Based Speech Emotion Recognition System Using 1D CNN LSTM Networks

Speech Emotion Recognition (SER) is the task of recognizing a speaker’s emotional state from speech. SER plays a significant role in Human-Computer Interaction and psychological assessment. Several kinds of time-frequency representations like spectrograms, mel-frequency cepstrum coefficients (MFCCs), and mel-spectrograms are commonly used to develop an SER system. These representations use the Fast Fourier Transform (FFT) to convert the time domain signal to the frequency domain. However, the FFT has one fundamental limitation due to the uncertainty principle, which does not simultaneously allow a good resolution in both time and frequency domains. On the other hand, the multiresolution property of wavelets can provide a good localization in both time and frequency domains. Therefore, this article investigates the competency of the wavelet transforms for SER. We propose a Wavelet based Deep Emotion Recognition (WaDER) method using an autoencoder and 1D convolutional neural network (CNN) and long short-term memory (LSTM) networks. The autoencoder is used to perform the dimensionality reduction of the wavelet features then the latent space is used to classify the emotions using the 1D CNN-LSTM model. We conducted a Monte-Carlo K-fold validation using the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) dataset. For speaker-dependent (SD) experiments, we achieved an unweighted accuracy (UA) of 81.45% and a weighted accuracy (WA) of 81.22%. The results of the experiments on the RAVDESS dataset show that the proposed method performs better than the state-of-the-art methods, which use other time-frequency representations