LLM-as-a-tutor in EFL Writing Education: Focusing on Evaluation of Student-LLM Interaction
This work addresses the problem of assessing LLM tutors for EFL learners, but it appears incremental as it builds on existing LLM applications in education.
The paper tackled the challenge of evaluating LLM-as-a-tutor systems in EFL writing education by integrating pedagogical principles to assess student-LLM interaction, proposing three metrics and involving experts and learners in evaluation to lay groundwork for tailored LLM development.
In the context of English as a Foreign Language (EFL) writing education, LLM-as-a-tutor can assist students by providing real-time feedback on their essays. However, challenges arise in assessing LLM-as-a-tutor due to differing standards between educational and general use cases. To bridge this gap, we integrate pedagogical principles to assess student-LLM interaction. First, we explore how LLMs can function as English tutors, providing effective essay feedback tailored to students. Second, we propose three metrics to evaluate LLM-as-a-tutor specifically designed for EFL writing education, emphasizing pedagogical aspects. In this process, EFL experts evaluate the feedback from LLM-as-a-tutor regarding quality and characteristics. On the other hand, EFL learners assess their learning outcomes from interaction with LLM-as-a-tutor. This approach lays the groundwork for developing LLMs-as-a-tutor tailored to the needs of EFL learners, advancing the effectiveness of writing education in this context.