Grammatical Error Feedback: An Implicit Evaluation Approach
This addresses the need for holistic feedback evaluation in computer-assisted language learning, though it is incremental as it builds on existing GEC and LLM methods.
The paper tackles the problem of evaluating grammatical error feedback (GEF) for second language learning by introducing an implicit evaluation approach that uses a grammatical lineup method with LLMs, eliminating manual annotations and applied to the Cambridge Learner Corpus.
Grammatical feedback is crucial for consolidating second language (L2) learning. Most research in computer-assisted language learning has focused on feedback through grammatical error correction (GEC) systems, rather than examining more holistic feedback that may be more useful for learners. This holistic feedback will be referred to as grammatical error feedback (GEF). In this paper, we present a novel implicit evaluation approach to GEF that eliminates the need for manual feedback annotations. Our method adopts a grammatical lineup approach where the task is to pair feedback and essay representations from a set of possible alternatives. This matching process can be performed by appropriately prompting a large language model (LLM). An important aspect of this process, explored here, is the form of the lineup, i.e., the selection of foils. This paper exploits this framework to examine the quality and need for GEC to generate feedback, as well as the system used to generate feedback, using essays from the Cambridge Learner Corpus.