Learning and understanding partner dance through motion analysis in a virtual environment

Partner dance is a physical activity that is social and involves two partners dancing to music. Learning such dance is difficult and presents many challenges, such as finding a partner, learning the right skills, etc. Through the motion analysis of couples as entities with regards to tempos, we aim at developing new ways to understand and characterize partner dance. We are proposing a set of features that can extract high-level information from low-level data, usable enough to help design learning systems. In the first place, we selected three main dance skills considered important for Salsa dance learning from interviews with experts - namely Rhythm, Guidance, and Styling - and searched for clues in the theoretical description of the movement that could help to describe high-level dance skills in terms of low-level motion measurements. Next, we suggest a set of musicalrelated motion features (MMF) to characterize Salsa performance (two people interacting and dancing with each other to the music) in terms of learning levels. These features lead to 29 measurements applied to captured motion data and music beat analysis. Secondly, a database is created with 26 couples performing from three levels of experience (beginners: less than 6 months, intermediates: 1 to 3 years, experts: more than five years - teachers or jury) along with ten music with varying tempos. All MMFs are then extracted from the database and submitted to a machine learning algorithm. The "Random forest" algorithm shows the best results for level classification, up to 90% accuracy. A feature importance study is also conducted to enhance our MMF set by identifying the features that contribute the most to the classification. Finally, we designed and evaluated an interactive dance learning system that allows a person to learn Guidance and Rhythm skills through a virtual reality Salsa dance environment containing a virtual partner with hand to hand interaction. We installed the experiment in different dance schools and at dance parties to make several dancers of various levels test our system. The overall user motion data was recorded and a learning analysis performed using our Musical Motion Features and the well-known Laban Motion Analysis, showing an improvement of the dance skills of users after training, thus validating the relevance of our design. More extended studies could be conducted to further validate the learning aspect of our interactive system by defining more precisely the learning groups, and new exercises could be imagined to add new skills to the existing rhythm and guidance. Our proposed framework of MMFs could also be refined by considering the more complex and precise definitions of each of the features, taking more skeleton joints into account than the foot joints for the Rhythm skill. The Guidance skill can also be redefined in the light of the final design of our learning system, based on gesture recognition on a specific time frame. Overall, our approach, rooted in the reality of current Salsa dance social performance, shows to be relevant for learning state classification and for improving dance skills with an innovative immersive virtual reality environment