K.: Collaborative Capturing and Interpretation of Interactions

This paper proposes a notion of interaction corpus, a captured collection of human behaviors and interactions among humans and artifacts. Digital multimedia and ubiquitous sensor technologies create a venue to capture and store interactions that are automatically annotated. A very large-scale accumulated corpus provides an im-portant infrastructure for a future digital society for both humans and computers to understand verbal/non-verbal mechanisms of human interactions. The interac-tion corpus can also be used as a well-structured stored experience, which is shared with other people for com-munication and creation of further experiences. Our approach employs wearable and ubiquitous sensors, such as video cameras, microphones, and tracking tags, to capture all of the events from multiple viewpoints simul-taneously. We demonstrate an application of generating a video-based experience summary that is reconfigured automatically from the interaction corpus