Emotion Detection Using an Ensemble Model Trained with Physiological Signals and Inferred Arousal-Valence States

Affective computing is an exciting and transformative field that is gaining in popularity among psychologists, statisticians, and computer scientists. The ability of a machine to infer human emotion and mood, i.e. affective states, has the potential to greatly improve human-machine interaction in our increasingly digital world. In this work, an ensemble model methodology for detecting human emotions across multiple subjects is outlined. The Continuously Annotated Signals of Emotion (CASE) dataset, which is a dataset of physiological signals labeled with discrete emotions from video stimuli as well as subject-reported continuous emotions, arousal and valence, from the circumplex model, is used for training and testing the model [1, 2]. Blood volume pulse (BVP), galvanic skin response (GSR), and skin temperature physiological signals are windowed and used to extract 17 physiological features (13 BVP, 2 GSR, and 2 skin temperature features). These physiological features are then used along with subject reported arousal and valence state values as inputs into regression models to create predicted arousal and valence values for each feature window. The predicted or “inferred” arousal and valence state values were then concatenated to the original 17 physiological features and used as inputs to a classification model for the final classification of emotion state into five categories, including relaxed, bored, neutral, amused, and scared. Multiple regression and classification models were tested, and the best performing model was a linear regression arousal and valence predictor followed by a hyperparameter-tuned support vector machine (SVM) classifier, achieving a five-fold cross-validation accuracy of 98.79% ± 0.29% for the five-class emotion classification across subjects. Finally, an impactful real-world application in an emotional feedback household environment for enabling independent living in differently-abled people is discussed