Improving Social Communication Skills Using Kinesics Feedback

Interactive training environments typically include feedback mechanisms designed to help trainees improve their performance through guided or self-reflection. When the training system deals with human-to-human communications, as one would find in a teacher, counselor or cross-cultural trainer, such feedback needs to focus on all aspects of human communication. This means that, in addition to verbal communication, nonverbal messages (kinesics in particular) must be captured and analyzed for semantic meaning. The goal of this research is to introduce interactive training models developed to improve human-to-human interaction. The specific context in which we prototype and validate these models is the TeachLivE™ teacher rehearsal environment developed at the University of Central Florida. We implemented an online gesture recognition application on top of the Microsoft Kinect software development kit with multiple feedback channels including visual and haptics. In a study of twelve participants rehearsing a teaching session in TeachLivE, we found that the online gesture recognition tool and its associated feedback method are effective and non-intrusive approaches for the purpose of communication-skill training. The algorithms employed, the results, and the implications for other interactive contexts are discussed in this paper