Piano Crossing - Generating music using computer vision

Piano Crossing is an interactive art installation which transforms the zebra crossing into a working piano keyboard where pedestrians generate music by walking across it. Installation consists of the zeboard - zebra crossing turned keyboard - by adding black stripes to the existing white ones, serving as keys, a digital camera overlooking the zeboard and of a laptop computer with speakers. The computer runs a specially designed software, which recognizes pedestrian's contacts with the keys by using OpenCV's computer vision methods, and by that generates a MIDI sequence, which furthermore turns into music. The application consists of two main processes. 'Generate Keyboard' is about creating abstract presentation of MIDI keyboard from an input frame. The image is segmentated to individual regions belonging to the keys. Each key includes a contour of the region, a MIDI note number and a key status. The abstract keyboard presentation is then used in the 'Track and Play' process, which brings the actual execution. For the pedestrian positioning two different background substraction techniques can be used; the simple technique or the advanced codebook model technique. The installation software was tested on two different models of the Piano Crossing; on an actual zebra crossing outdoor and on the reduced-in-size model indoor