Considerations for Dealing with Real-Time Communications in an Intelligent Team Tutoring System Experiment

In our paper we discuss an intelligent team tutoring system (ITTS) developed for a computer-based surveillance task experiment. In the experiment, two teammates worked together in a shared Virtual Battlespace 2 environment, and tutoring was provided by the Generalized Intelligent Framework for Tutoring. Feedback was triggered by the actions that the team members took, which were largely communications based. Since natural language processing software was not available in the tutor, we devised ways to measure and react to communications in real time. Team members both pushed keyboard buttons associated with the message they intended to send (which was processed by the computer and used to drive feedback) and were asked to verbally speak the same information to their teammates. In this paper, we discuss the reasoning behind the process used, challenges associated with using real-time communication in an ITTS, and initial analysis approaches that were done using the audio recordings of teammates\u27 sessions. Emphasis is placed on how to reconcile real-time inputs to the computer system with audio recordings that occurred during the session and were later used for analysis. We discuss the challenges we encountered engaging with a real-time ITTS, which relies on communications between team members, and provide suggestions on addressing these challenges for future experiments