Lessons Learned in Exploring the Leap Motion(TM) Sensor for Gesture-based Instrument Design

The Leap Motion(TM) sensor offers fine-grained gesture-recognition and hand tracking. Since its release, there have been several uses of the device for instrument design, musical interaction and expression control, documented through online video. However, there has been little formal documented investigation of the potential and challenges of the platform in this context. This paper presents lessons learned from work-in-progress on the development of musical instruments and control applications using the Leap Motion(TM) sensor. Two instruments are presented: Air-Keys and Air-Pads and the potential for augmentation of a traditional keyboard is explored. The results show that the platform is promising in this context but requires various challenges, both physical and logical, to be overcome