Exploring different movement sonification strategies for rehabilitation in clinical settings

International audienceWe describe an interactive system that allows for sonifying arm movements. The aim is to support stroke patients going through rehabilitation by providing them with augmented auditory feedback that reacts to their movements. The system is based on IMU sensors (Inertial Measurements Unit) attached to each arm. The movement data are streamed in real-time to a laptop computer that generates various sounds or musical interactions using a program we developed. We tested different types of auditory feedback, each of them using a specific strategy for the sound-movement mapping. The first type of movement-sound mappings is based on direct relationships between the reaching distance and either the pitch of a continuous tone, or the tempo of a regular beat pattern. The second type of mapping is music-oriented: the user movement allows for controlling the tempo of musical pieces. The third type of mapping associates the hand position to specific environmental sounds. We report here on the technical system along with preliminary results in a clinical setting with both post-stroke patients and healthy users. CCS CONCEPTS • Human-centered computing → Sound-based input / output; Auditory feedback; Gestural input; Usability testing; Interaction devices