Automated Essay Scoring based on Two-Stage Learning
This addresses a robustness issue in automated essay scoring for educational applications, but it is incremental as it builds on existing methods.
The paper tackles the problem of automated essay scoring models being vulnerable to adversarial samples like permuted sentences and irrelevant essays, and develops a Two-Stage Learning Framework (TSLF) that integrates feature-engineered and end-to-end approaches, achieving new state-of-the-art average performance on five-eighths of prompts and showing great robustness against adversarial essays.
Current state-of-art feature-engineered and end-to-end Automated Essay Score (AES) methods are proven to be unable to detect adversarial samples, e.g. the essays composed of permuted sentences and the prompt-irrelevant essays. Focusing on the problem, we develop a Two-Stage Learning Framework (TSLF) which integrates the advantages of both feature-engineered and end-to-end AES models. In experiments, we compare TSLF against a number of strong baselines, and the results demonstrate the effectiveness and robustness of our models. TSLF surpasses all the baselines on five-eighths of prompts and achieves new state-of-the-art average performance when without negative samples. After adding some adversarial essays to the original datasets, TSLF outperforms the feature-engineered and end-to-end baselines to a great extent, and shows great robustness.