Influence of Music on Perceived Emotions in Film

"Influence of Music on Perceived Emotions in Film" has received an Honorable Mention award at the AES 153rd Convention. Honorable Mention awards are given only to the top 3 papers in overall reviewer score with additional input from the AES publications committee.International audienceFilm music plays a core role in film production and reception as it not only contributes to the film aesthetics and creativity, but it also affects viewers’ experience and enjoyment. Film music composers often aim to serve the film narrative, immerse viewers into the setting and story, convey clues, and importantly, act on their emotions. Yet, how film music influences viewers is still misunderstood. We conducted a perceptual study to analyse the impact of music on the perception of emotions in film. We developed an online interface for time-based emotion annotation of audio/video media clips based on the Valence/Arousal (VA) two-dimensional model. Participants reported their perceived emotions over time in the VA space for three media conditions: film scene presented without sound (video only), film music presented without video (audio only), and film scene with accompanying music and sound effects (both video and audio modalities). 16 film clips were selected covering four clips for each of four genres (action & drama, romance, comedy, and horror). 38 participants completed the study (12 females and 26 males from many countries, average age: 28.9). Density scatter plots are used to visualise the spread of emotion ratings in the VA space and differences across media conditions and film clips. Results from linear mixed effect models show significant effects of the audiovisual media condition and film genre on VA ratings, in line with previous results by Parke et al. [1]. Perceived VA ratings across media conditions follow an almost linear relationship with an increase in strength in the following order: film alone, film with music/sound, music alone. We illustrate this effect by plotting the VA rating centre of mass across conditions. VA ratings for the film-alone condition are closer to the origin of the space compared to the two other media conditions, indicating that the inclusion of music yields stronger emotions (higher VA ratings). Certain individual factors (musical ability, familiarity, preference) also seem to impact the perception of arousal and valence while viewing films. Our online emotion annotation interface was on overall well received and suggestions to improve the display of reference emotion tags are discussed