An Automated Explainable Educational Assessment System Built on LLMs
This system addresses the need for explainable and cost-effective automated assessment tools for educators and researchers, though it appears incremental as it applies existing LLM methods to a specific domain.
The researchers tackled the problem of limited explainability and high annotation costs in automated educational assessment by developing AERA Chat, a system that uses large language models to automatically mark student responses and generate rationale explanations, providing educators with insights into assessment accuracy and rationale quality.
In this demo, we present AERA Chat, an automated and explainable educational assessment system designed for interactive and visual evaluations of student responses. This system leverages large language models (LLMs) to generate automated marking and rationale explanations, addressing the challenge of limited explainability in automated educational assessment and the high costs associated with annotation. Our system allows users to input questions and student answers, providing educators and researchers with insights into assessment accuracy and the quality of LLM-assessed rationales. Additionally, it offers advanced visualization and robust evaluation tools, enhancing the usability for educational assessment and facilitating efficient rationale verification. Our demo video can be found at https://youtu.be/qUSjz-sxlBc.