Human Nonverbal Behaviour Understanding in the Wild for New Media Art

Abstract. Over the course of the London 2012 Olympics a large public installation took place in Central London. Its premise was to enable mem-bers of the public to express themselves by controlling the lights around the rim of the London Eye. The installation’s design and development was undertaken as a collaborative project between an interactive arts studio and researchers in the field of affective and behavioural comput-ing. Over 800 people participated, taking control of the lights using their heart rates and hand gestures. This paper approaches nonverbal and af-fective behaviour understanding for new media art as a case study, and reports the design of this installation and the subsequent analysis of over one million frames of physiological and motion capture data. In doing so it sheds light on how the intersection of affective and behavioural com-puting and new media art could be beneficial to both researchers and artists