Phrase-Level Adversarial Training for Mitigating Bias in Neural Network-based Automatic Essay Scoring
This addresses bias and robustness issues in AES for educational evaluation, but it is incremental as it builds on existing adversarial training methods.
The study tackled bias and lack of robustness in Automatic Essay Scoring (AES) systems by proposing a phrase-level adversarial training method to generate an adversarial essay set, which significantly improved AES model performance in both adversarial and non-adversarial scenarios.
Automatic Essay Scoring (AES) is widely used to evaluate candidates for educational purposes. However, due to the lack of representative data, most existing AES systems are not robust, and their scoring predictions are biased towards the most represented data samples. In this study, we propose a model-agnostic phrase-level method to generate an adversarial essay set to address the biases and robustness of AES models. Specifically, we construct an attack test set comprising samples from the original test set and adversarially generated samples using our proposed method. To evaluate the effectiveness of the attack strategy and data augmentation, we conducted a comprehensive analysis utilizing various neural network scoring models. Experimental results show that the proposed approach significantly improves AES model performance in the presence of adversarial examples and scenarios without such attacks.