Polarity in the Classroom: A Case Study Leveraging Peer Sentiment Toward Scalable Assessment
This work addresses scalable assessment for instructors in massive open online courses, but it is incremental as it builds on existing peer review methods by adding sentiment analysis.
The authors tackled the problem of unreliable peer grading in large online courses by developing a sentiment analysis algorithm that uses student review comments to inform and validate grades, analyzing over 6800 peer reviews from nine courses to assess its viability.
Accurately grading open-ended assignments in large or massive open online courses (MOOCs) is non-trivial. Peer review is a promising solution but can be unreliable due to few reviewers and an unevaluated review form. To date, no work has 1) leveraged sentiment analysis in the peer-review process to inform or validate grades or 2) utilized aspect extraction to craft a review form from what students actually communicated. Our work utilizes, rather than discards, student data from review form comments to deliver better information to the instructor. In this work, we detail the process by which we create our domain-dependent lexicon and aspect-informed review form as well as our entire sentiment analysis algorithm which provides a fine-grained sentiment score from text alone. We end by analyzing validity and discussing conclusions from our corpus of over 6800 peer reviews from nine courses to understand the viability of sentiment in the classroom for increasing the information from and reliability of grading open-ended assignments in large courses.