Effect of Feature Extraction when Classifying Emotions in Speech - An Applied Study

The demand for machines that can interact with its users through speech is growing. For example, four of the world’s largest IT companies; Amazon, Apple, Google and Microsoft, are developing intelligent personal assistants who are able to communicate through speech. In this thesis, we have investigated the effect of feature extraction when classifying emotions in speech, using a convolutional neural network (CNN). We used the software openSMILE to extract two sets of features, and one set of raw data, from recorded audio, and compared the CNN’s classification accuracy of the sets with eight, five and three classes of emotions. We used one architecture of the CNN, to be fair when comparing each feature set, and implemented it using Keras. The CNN architecture was developed by an experimenta lapproach.The feature set that gave the highest accuracy managed to reach 39 % accuracy when classifying eight emotions (random guessing would yield around 12.5 % accuracy on average), 53 % with five emotions (compared to around 20 % if just guessing), and 69 % with threee motions (compared to around 33 % if just guessing). This set also performed best whendistinguishing emotions from each other.The result shows that using feature extraction improves the accuracy, but more features does not necessarily increase accuracy.While the classification accuracies in this study may seem low, it is important to remember that even for humans, it can be hard to distinguish different feelings based on just the pitch of other people’s voices