Automatic Detection of Inauthentic Templated Responses in English Language Assessments
This addresses cheating in high-stakes language tests, but it is incremental as it applies existing methods to a specific domain problem.
The study tackled the problem of low-skill test takers using memorized templates to deceive automated scoring systems in English Language Assessments by introducing the AuDITR task and a machine learning approach, emphasizing the need for model updates in production.
In high-stakes English Language Assessments, low-skill test takers may employ memorized materials called ``templates'' on essay questions to ``game'' or fool the automated scoring system. In this study, we introduce the automated detection of inauthentic, templated responses (AuDITR) task, describe a machine learning-based approach to this task and illustrate the importance of regularly updating these models in production.