Design of an Automated System for the Retrieval of Emotional Content in Natural Images

Emotions often drive us and our behaviors. An ability to reliably and automatically estimate the degree and kind of emotion aroused in a typical individual has widespread applicability. Two dimensions of emotion are key: arousal, or the extent to which a scene excites/calms and its valence, i.e. the extent to which it is pleasant/unpleasant. Past attempts to automate emotion detection in images have failed. Here, we adopt a hybrid, integrated approach that broadly consists of two components: a front-end consisting of a bank of classifiers that recognize the presence of specific semantic content (e.g. person/animal/beach) in the image; a back-end that weights the importance of each semantic category to generate a discrete output of image valence and arousal. Model performance was compared with ground truth, i.e. ratings provided by human subjects: test accuracy was 96.0% on valence (chance=50%) and 92.0% on arousal classification (chance=33%) across the externally validated image set.Electrical and Computer Engineering, Department o