Designing and Evaluating Hint Generation Systems for Science Education
This work addresses the challenge of hindering conceptual understanding in science education for secondary students, though it is incremental in exploring hinting strategies.
The study tackled the problem of large language models giving away answers in education by designing hint generation systems to promote active learning, finding through a quantitative study with 41 participants that learners have different preferences for static versus dynamic hints and that automatic metrics fail to capture these preferences.
Large language models are influencing the education landscape, with students relying on them in their learning process. Often implemented using general-purpose models, these systems are likely to give away the answers, which could hinder conceptual understanding and critical thinking. We study the role of automatic hint generation as a pedagogical strategy to promote active engagement with the learning content, while guiding learners toward the answers. Focusing on scientific topics at the secondary education level, we explore the potential of large language models to generate chains of hints that scaffold learners without revealing answers. We compare two distinct hinting strategies: static hints, pre-generated for each problem, and dynamic hints, adapted to learners' progress. Through a quantitative study with 41 participants, we uncover different preferences among learners with respect to hinting strategies, and identify the limitations of automatic evaluation metrics to capture them. Our findings highlight key design considerations for future research on hint generation and intelligent tutoring systems that seek to develop learner-centered educational technologies.