Did That Help? Harnessing Natural Language Processing to Uncover Characteristics of Peer Feedback and Their Impacts on Teamwork-Skills Learning.

Peer Evaluation Systems (PESs) allow members of student teams to provide one another with computer-mediated feedback in the form of qualitative, open-ended comments. The current research leverages unsupervised Natural Language Processing (NLP), namely Biterm Topic Modeling (BTM) and sentiment analysis, to uncover latent topics and degree of positivity and negativity expressed in peer feedback, respectively. BTM results revealed a 6-topic model that was reliably replicated over 10 Gibbs initializations 80% of the time. Topics were labeled Timely Communication, Idea Generation, Coordination & Adaptation, Work Quality, Team Support & Focusing, and Work Accountability. Qualitative comparison suggests that these topics demonstrate significant overlap with concepts detailed within existing teamwork and feedback frameworks. Sentiment analysis indicated that peer feedback had a predominantly neutral to positive valence orientation, but that the analysis had limited accuracy. Significance testing evaluating the impact of the topic of feedback, feedback valence, and feedback length on outcome measures of students’ learning of teamwork skills were entirely non-significant. These results are discussed, along with discussions of NLP’s potential to expand existing theories and frameworks with data-driven techniques, and to provide educators with rapid, high-level insight from PESs that support student learning outcomes