Annotation and Classification of Sentence-level Revision Improvement
This addresses the challenge of evaluating writing revisions for educators and researchers, though it is incremental as it builds on existing datasets and methods.
The researchers tackled the problem of assessing revision quality in writing by creating a corpus of student essay revisions annotated for improvement, then developed a machine learning model to predict revision improvement, finding that blending expert and non-expert revisions increased model performance, with expert data particularly boosting prediction of low-quality revisions.
Studies of writing revisions rarely focus on revision quality. To address this issue, we introduce a corpus of between-draft revisions of student argumentative essays, annotated as to whether each revision improves essay quality. We demonstrate a potential usage of our annotations by developing a machine learning model to predict revision improvement. With the goal of expanding training data, we also extract revisions from a dataset edited by expert proofreaders. Our results indicate that blending expert and non-expert revisions increases model performance, with expert data particularly important for predicting low-quality revisions.