Evaluating ChatGPT's Decimal Skills and Feedback Generation in a Digital Learning Game
This addresses the challenge of automated grading and feedback for unconstrained student input in educational technology, though it is incremental in applying existing AI models to a specific domain.
The study evaluated ChatGPT's ability to handle open-ended self-explanations in a digital learning game for decimals, finding it accurately assessed 75% of student answers and generated high-quality feedback similar to human instructors, but struggled with decimal place values and number line problems.
While open-ended self-explanations have been shown to promote robust learning in multiple studies, they pose significant challenges to automated grading and feedback in technology-enhanced learning, due to the unconstrained nature of the students' input. Our work investigates whether recent advances in Large Language Models, and in particular ChatGPT, can address this issue. Using decimal exercises and student data from a prior study of the learning game Decimal Point, with more than 5,000 open-ended self-explanation responses, we investigate ChatGPT's capability in (1) solving the in-game exercises, (2) determining the correctness of students' answers, and (3) providing meaningful feedback to incorrect answers. Our results showed that ChatGPT can respond well to conceptual questions, but struggled with decimal place values and number line problems. In addition, it was able to accurately assess the correctness of 75% of the students' answers and generated generally high-quality feedback, similar to human instructors. We conclude with a discussion of ChatGPT's strengths and weaknesses and suggest several venues for extending its use cases in digital teaching and learning.