Exploring the Value of Diverse LLM Explanations in Introductory Programming
For computing educators, this work suggests that using diverse LLM explanations can enhance student learning without additional cognitive burden, though the effect is modest.
The study investigates whether providing multiple diverse LLM-generated explanations (emphasizing different aspects like function, concept, goal) improves student understanding in introductory programming compared to generic explanations. Results show that diverse explanations led to about 7.7% higher accuracy on open-ended questions with no increase in perceived cognitive load.
Large Language Models (LLMs) have shown the potential to generate code explanations that surpass those of peers in quality, offering promising opportunities for computer science education. While these explanations may not yet match the depth and clarity of instructor-provided explanations, research in computational creativity highlights that the quantity and diversity of ideas can often outweigh a singular focus on quality. Inspired by this, we explore whether combining multiple diverse explanations, each emphasizing distinct aspects (e.g., function, concept, goal), can enhance students' understanding of programming exercises compared to generic explanations that do not emphasize distinct conceptual aspects. In our study 971 first-year computing students were randomly assigned either diverse or generic LLM-generated explanations for two programming exercises. Students completed multiple-choice and open-ended questions for each exercise, followed by Likert-scale questions and open-ended reflections. Our findings outline patterns in student performance and perceived cognitive load across the two explanation conditions. These findings highlight how variation in explanation emphasis may relate to learner engagement and understanding. Across participants, open-ended response accuracy was consistently about 7.7% higher when students received diverse explanations, with no difference in perceived cognitive load.