A Picture is Worth a Thousand Songs: Exploring Visual Aspects of Music

The abstract nature of music makes it intrinsically hard to describe. To alleviate this problem we use well known songs or artists as a reference to describe new music. Music infor-mation research has mimicked this behavior by introducing search systems that rely on prototypical songs. Based on similarity models deducing from signal processing or collab-orative filtering an according list of songs with similar prop-erties is retrieved. Yet, music is often searched for a specific intention such as music for workout, to focus or for a com-fortable dinner with friends. Modeling music similarities based on such criteria is in many cases problematic or vi-sionary. Despite these open research challenges a more user focused question should be raised: Which interface is ade-quate for describing such intentions? Traditionally queries are either based on text input or seed songs. Both are in many cases inadequate or require extensive interaction or knowledge from the user. Despite the multi-sensory capabilities of humans, we pri-marily focus on vision. Many intentions for music searches can easily be pictorially envisioned. This paper suggests to pursue a query music by image approach. Yet, exten-sive research in all disciplinary fields of music research, such as music psychology, musicology and information technolo-gies, is required to identify correlations between the acoustic and the visual domain. This paper elaborates on opportuni-ties and obstacles and proposes ways to approach the stated problems. 1