Developing humanoids for musical interaction

Abstract — For many people, playing and enjoying music are integral activities in their daily lives, and the development of musically-aware robot systems provides a unique opportunity for richer forms of human-robot interaction. A robot partic-ipating in an ensemble musical performance requires a wide variety of skills in order to perform alongside humans. For instance, human musicians make use of substantial auditory and visual information throughout a performance; they watch the conductor and other musicians for rhythm and other musical cues, and they listen to their own instrument as well as the overall ensemble in order to adjust their own performance. Our current work focuses on providing such capabilities (e.g., audio and visual beat detection, note onset and pitch detection, and basic control for music keyboard performance), with the long-term goal of enabling a large humanoid to be an interactive participant in a live music ensemble. We use miniature humanoids to first prototype and refine many of these systems before deploying them on the life-sized KAIST Hubo humanoid. I