Sentiment Analysis in Peer Review

Sentiment analysis, a widely popular subfield of natural language processing, has recently been used in the classroom to predict student attrition or to determine the mood of students, teacher strengths and weaknesses, or student perception of internship experience. These are all helpful indicators for the enhancement of students\u27 academic experience but none improve the information gathered from or the reliability of peer review. This is particularly important in large courses with complex assignments (e.g., essays, software projects, and presentations) where scalable grading is requisite. In this dissertation, we apply sentiment analysis not on an assignment itself, but on the meaningful content generated by a learning, peer-reviewing crowd to produce a fine-grained, quantitative score from peer review text alone. To obtain a reliable score from peer review text, we first supply an educational framework that increases the amount of critical feedback students provide. We then utilize an aspect extractor to aggregate pertinent information from student content and modify our review form in a data-driven, iterative fashion. HeLPS, our domain-specific lexicon, was mined from peer review comments and exhibits high precision on peer review text compared to other publicly-available lexicons. Our sentiment analysis algorithm, SentiSoft, leverages both the lexicon and aspect extractor to provide a fine-grained sentiment score with metrics and supporting documentation from text alone. The combination of sentiment analysis on text and an iteratively-refined review form improves our ability to understand student feedback and ultimately facilitates scalable assessment