Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input
This addresses a security and validity issue for educational technology and AI systems that rely on automated essay scoring, though it is an incremental improvement over existing methods.
The paper tackled the problem of automated essay scoring (AES) models being vulnerable to adversarially crafted, grammatical but incoherent input, and developed a neural coherence model integrated with AES to improve scoring and detect such adversarial sequences, showing effectiveness in experiments.
We demonstrate that current state-of-the-art approaches to Automated Essay Scoring (AES) are not well-suited to capturing adversarially crafted input of grammatical but incoherent sequences of sentences. We develop a neural model of local coherence that can effectively learn connectedness features between sentences, and propose a framework for integrating and jointly training the local coherence model with a state-of-the-art AES model. We evaluate our approach against a number of baselines and experimentally demonstrate its effectiveness on both the AES task and the task of flagging adversarial input, further contributing to the development of an approach that strengthens the validity of neural essay scoring models.