Investigating Conversational Agents to Support Secondary School Students Learning CSP
This work addresses the need for better learning support tools for high school students in CSP courses, but the findings are incremental and limited to a small-scale study.
The study evaluated the potential of conversational agents (both general-purpose generative like ChatGPT and custom fixed-response) to help secondary school students learn AP Computer Science Principles concepts, based on classroom use by 45 students. Results showed that custom agents were more effective for targeted learning, while generative agents provided broader but less reliable support.
Secondary school students enrolled in the AP Computer Science Principles (CSP) course commonly utilize web resources (e.g., tutorials, Q\&A sites) to better understand key concepts in the curriculum. The primary obstacle to using these resources is finding information appropriate for the learning task and student's background. In addition to web search, conversational agents are increasingly a viable alternative for CSP students. In this paper, we study the potential of conversational agents to aid secondary school students as they acquire knowledge on CSP concepts. We explore general purpose, generative conversational agents (e.g., ChatGPT) and custom, fixed-response conversational agents built specifically to aid CSP students. We present results from classroom use by 45 high school students in grades 9-11 (ages 14-17) across six CSP sections. Our main contributions are in better understanding how conversational agents can help CSP students and an evaluation of the effectiveness and engagement of different approaches for CSP exploratory search.