Detecting Off-topic Responses to Visual Prompts
This addresses a weakness in automated essay scoring for language learners by incorporating visual prompts, but it is incremental as it builds on existing text-based relevance detection methods.
The paper tackled the problem of detecting off-topic responses to visual prompts in automated essay scoring, proposing a neural architecture that was evaluated on a dataset of texts from language learners.
Automated methods for essay scoring have made great progress in recent years, achieving accuracies very close to human annotators. However, a known weakness of such automated scorers is not taking into account the semantic relevance of the submitted text. While there is existing work on detecting answer relevance given a textual prompt, very little previous research has been done to incorporate visual writing prompts. We propose a neural architecture and several extensions for detecting off-topic responses to visual prompts and evaluate it on a dataset of texts written by language learners.