Improving Machine Recognition of Collaborative Dialog Acts via Sentence Embeddings

This project attempts to train a computer to recognize linguistic dialogue acts within transcripts of students working together. In COMPS (Computer-Mediated Problems Solving) exercises students work together via typed-chat, solving problems in small groups in a computer science class. Student dialogue turns are classified according to four categories of collaborative utterance: sharing ideas, negotiating ideas, regulating problem-solving, and maintaining communication. Previous work constructed machine classifiers to recognize these four acts. It achieved F1 accuracy (a combination of precision and recall) ranging from 0.6 for recognizing sharing-ideas, to 0.3 for maintaining-conversation. This project attempts to improve the accuracy. The first approach is to preprocess the text with doc2vec, replacing topic modeling. Topic modeling treats sentences as unordered bags of words. In the new approach, two different sentences containing the same words will appear differently to the classifier, making it more likely that the software will be able to recognize the different conversational intentions of the speakers. Another approach will train the classifier utilizing the dialogue context of several previous turns. Patterns of collaborative dialogue acts are expected to reveal the conversational fingerprints on how students collaborate. This research advances toward promoting better student collaborative problem-solving exercises, more fully using student group cognition and collaboration skills, and potentially developing computer-monitoring of the student conversation groups