FEANEL: A Benchmark for Fine-Grained Error Analysis in K-12 English Writing
This addresses the problem of providing detailed educational feedback for K-12 English learners, though it is incremental as it focuses on benchmarking rather than new methods.
The paper introduced the FEANEL benchmark for fine-grained error analysis in K-12 English writing, comprising 1,000 annotated essays, and found significant gaps in current LLMs' ability to perform this task.
Large Language Models (LLMs) have transformed artificial intelligence, offering profound opportunities for educational applications. However, their ability to provide fine-grained educational feedback for K-12 English writing remains underexplored. In this paper, we challenge the error analysis and pedagogical skills of LLMs by introducing the problem of Fine-grained Error Analysis for English Learners and present the Fine-grained Error ANalysis for English Learners (FEANEL) Benchmark. The benchmark comprises 1,000 essays written by elementary and secondary school students, and a well-developed English writing error taxonomy. Each error is annotated by language education experts and categorized by type, severity, and explanatory feedback, using a part-of-speech-based taxonomy they co-developed. We evaluate state-of-the-art LLMs on the FEANEL Benchmark to explore their error analysis and pedagogical abilities. Experimental results reveal significant gaps in current LLMs' ability to perform fine-grained error analysis, highlighting the need for advancements in particular methods for educational applications.