Investigating User Collaboration in Music Based Games

This study uses a combined method for the analysis of social interplay/interaction among users (or players) in a multimodal interaction and musical performance situation. The combined method consists of a) realtime interface data analysis for the description and interpretation of player actions detected by the system and b) video analysis used to describe and interpret the interaction situation and the context in which the social interplay takes place. This combined method is used in an iterative process, where the design of interactive games with musical-sonic feedback is improved according to newly discovered understandings and interpretations of joint user action. For example: How do two people play together if they play music with a pen tablet interface? Can a sound based computer game encourage two players to co-perform and co-create music? This study investigated two players’ joint performance in relation to their mutual play speed, synchronization and mirroring of play styles when ‘drawing with sound’.