Automated Formative Feedback for Short-form Writing: An LLM-Driven Approach and Adoption Analysis
This addresses the problem of scalable, personalized feedback for students and professors in educational settings, but it is incremental as it applies existing LLM methods to a specific domain with limited adoption.
The paper tackled the challenge of providing automated formative feedback for short-form student writing in an engineering Capstone program, developing an LLM-powered tool that improved report completeness and quality for engaged students, though adoption rates were low initially.
This paper explores the development and adoption of AI-based formative feedback in the context of biweekly reports in an engineering Capstone program. Each student is required to write a short report detailing their individual accomplishments over the past two weeks, which is then assessed by their advising professor. An LLM-powered tool was developed to provide students with personalized feedback on their draft reports, guiding them toward improved completeness and quality. Usage data across two rounds revealed an initial barrier to adoption, with low engagement rates. However, students who engaged in the AI feedback system demonstrated the ability to use it effectively, leading to improvements in the completeness and quality of their reports. Furthermore, the tool's task-parsing capabilities provided a novel approach to identify potential student organizational tasks and deliverables. The findings suggest initial skepticism toward the tool with a limited adoption within the studied context, however, they also highlight the potential for AI-driven tools to provide students and professors valuable insights and formative support.