Span Identification of Epistemic Stance-Taking in Academic Written English
This work addresses the need for automated writing evaluation systems to assess language use beyond lexis and grammar, focusing on academic English writing, but it is incremental as it builds on existing discourse-analytic frameworks and machine learning methods.
The paper tackled the problem of automatically identifying rhetorical stance expressions in academic English writing for automated writing evaluation, achieving a macro-averaged F1 score of 0.7208 with a RoBERTa + LSTM model, slightly outperforming intercoder reliability estimates.
Responding to the increasing need for automated writing evaluation (AWE) systems to assess language use beyond lexis and grammar (Burstein et al., 2016), we introduce a new approach to identify rhetorical features of stance in academic English writing. Drawing on the discourse-analytic framework of engagement in the Appraisal analysis (Martin & White, 2005), we manually annotated 4,688 sentences (126,411 tokens) for eight rhetorical stance categories (e.g., PROCLAIM, ATTRIBUTION) and additional discourse elements. We then report an experiment to train machine learning models to identify and categorize the spans of these stance expressions. The best-performing model (RoBERTa + LSTM) achieved macro-averaged F1 of .7208 in the span identification of stance-taking expressions, slightly outperforming the intercoder reliability estimates before adjudication (F1 = .6629).