AI-Tutoring in Software Engineering Education
This research addresses the need to understand student interactions with AI-Tutors in software engineering education, though it is incremental as it explores an emerging application without major breakthroughs.
The paper tackled the problem of evaluating Large Language Models as AI-Tutors in Automated Programming Assessment Systems by conducting an exploratory case study with GPT-3.5-Turbo integrated into Artemis, identifying user interaction patterns and highlighting advantages like timely feedback and challenges such as generic responses and concerns about learning progress inhibition.
With the rapid advancement of artificial intelligence (AI) in various domains, the education sector is set for transformation. The potential of AI-driven tools in enhancing the learning experience, especially in programming, is immense. However, the scientific evaluation of Large Language Models (LLMs) used in Automated Programming Assessment Systems (APASs) as an AI-Tutor remains largely unexplored. Therefore, there is a need to understand how students interact with such AI-Tutors and to analyze their experiences. In this paper, we conducted an exploratory case study by integrating the GPT-3.5-Turbo model as an AI-Tutor within the APAS Artemis. Through a combination of empirical data collection and an exploratory survey, we identified different user types based on their interaction patterns with the AI-Tutor. Additionally, the findings highlight advantages, such as timely feedback and scalability. However, challenges like generic responses and students' concerns about a learning progress inhibition when using the AI-Tutor were also evident. This research adds to the discourse on AI's role in education.