Key Phrase Classification in Complex Assignments
This work addresses the challenge of identifying important content for peer and expert feedback in educational settings, but it is incremental as it focuses on an empirical study of existing methods on a new dataset.
The paper tackled the problem of key phrase classification in complex educational assignments to improve feedback quality, finding that both pretrained language models and TF-IDF SVM classifiers yield similar results, with language models achieving an average 0.6 F1 higher.
Complex assignments typically consist of open-ended questions with large and diverse content in the context of both classroom and online graduate programs. With the sheer scale of these programs comes a variety of problems in peer and expert feedback, including rogue reviews. As such with the hope of identifying important contents needed for the review, in this work we present a very first work on key phrase classification with a detailed empirical study on traditional and most recent language modeling approaches. From this study, we find that the task of classification of key phrases is ambiguous at a human level producing Cohen's kappa of 0.77 on a new data set. Both pretrained language models and simple TFIDF SVM classifiers produce similar results with a former producing average of 0.6 F1 higher than the latter. We finally derive practical advice from our extensive empirical and model interpretability results for those interested in key phrase classification from educational reports in the future.