Automatically predicting mood from expressed emotions

Affect-adaptive systems have the potential to assist users that experience systematically negative moods. This thesis aims at building a platform for predicting automatically a person’s mood from his/her visual expressions. The key word is mood, namely a relatively long-term, stable and diffused affective state, as opposed to the short-term, volatile and intense emotion. This is emphasized, because mood and emotion often tend to be used as synonyms. However, since their differences are well established in the psychological literature, to address the thesis objective, we consider mood recognition as a different problem from emotion recognition. The main and key idea is to discover whether using the expressed emotions of a person over time can help us estimate the mood. The advantage of this notion is that we can reuse the bulk of work on automatic emotion recognition and plug in the mood recognition module. A big part of this thesis focuses on unveiling a functional relationship between the expressed emotions and mood. We first set out to verify whether simple aggregation rules, such as the average emotion, are good approximation of the mood. We continue by building incrementally more complex models that fit better the peculiarities of the mood estimation function. Equally important to retrieving the mood function, is to validate it on the proper data. We dedicate special attention on crafting a database that contains sufficiently long videos to capture the mood of a person, expressed subtly through genuine expressions. We annotate the expressed moods in the videos through crowdsourcing, as a fast way to obtain multiple ground truth labels per video. Our devised mood model and the publicly available database set a strong basis towards the development of the envisioned affect-assistive system, and more broadly towards the research on automatic mood recognition from videos.Intelligent SystemsElectrical Engineering, Mathematics and Computer Scienc