Jennifer M. Reddig

Jennifer M. Reddig, Glen R. Smith, Sanaz Ahmadzadeh Siyahrood et al.

AI-powered educational technologies have demonstrated measurable benefits for learners, but their design and evaluation have largely centered on K-12 contexts. As a result, many AI-supported learning systems remain poorly aligned with the needs, constraints, and goals of adult learners. To better understand how AI systems function in adult education, this paper examines the deployment of several AI learning technologies developed within a multidisciplinary, national research institute in the United States focused on adult learning and online education. Drawing on longitudinal deployment data, we conducted a reflexive thematic analysis to identify recurring challenges and design considerations across systems. These insights were synthesized into a set of 19 design guidelines intended to inform future AI-supported adult learning technologies. We demonstrate the utility of these guidelines through a heuristic evaluation of the deployed systems. Lastly, we present a guideline exploration tool that aids in the ideation of technologies by connecting the guidelines to stakeholder statements surfaced in the analysis process.

Jennifer M. Reddig, Scott Moon, Kaitlyn Crutcher et al.

As artificial intelligence (AI) becomes increasingly integrated into daily life, higher education must move beyond code-centric instruction to foster holistic AI literacy. We present a novel pedagogical approach that integrates embodied, unplugged activities into a university-level Introduction to AI course. Inspired by the effectiveness of CS Unplugged in K-12 education, our physical, collaborative activities gave students a first-person perspective on AI decision-making. Through interactive games modeling Search Algorithms, Markov Decision Processes, Q-learning, and Hidden Markov Models, students built an intuition for complex AI concepts and more easily transitioned to mathematical formalizations and code implementations. We present four unplugged AI activities, describe how to bridge from unplugged activities to plugged coding tasks, reflect on implementation challenges, and propose refinements. We suggest that unplugged activities can effectively bridge conceptual reasoning and technical skill-building in university-level AI education.

Jennifer M. Reddig, Scott Moon, Kaitlyn Crutcher et al.

Introductory artificial intelligence (AI) courses present significant learning challenges due to abstract concepts, mathematical complexity, and students' diverse technical backgrounds. While active and collaborative pedagogies are often recommended, implementation can be difficult at scale due to large class sizes and the intensive design effort required of instructors. This paper presents a quasi-experimental case study examining the redesign of in-class instructional time in a university-level Introduction to Artificial Intelligence course. Inspired by CS Unplugged approaches, we redesigned the summer offering, integrating embodied, unplugged simulations, collaborative programming labs, and structured reflection to provide students with a first-person perspective on AI decision-making. We maintained identical assignments, exams, and assessments as the traditional lecture-based offering. Using course evaluation data, final grade distributions, and post-course interviews, we examined differences in student engagement, experiences, and traditional learning outcomes. Quantitative results show that students in the redesigned course reported higher attendance, stronger agreement that assessments measured their understanding, and greater overall course effectiveness, despite no significant differences in final grades or self-reported learning. Qualitative findings indicate that unplugged simulations and collaboration fostered a safe, supportive learning environment that increased engagement and confidence with AI concepts. These results highlight the importance of in-class instructional design in improving students' learning experiences without compromising rigor.

Adit Gupta, Jennifer Reddig, Tommaso Calo et al.

Researchers have made notable progress in applying Large Language Models (LLMs) to solve math problems, as demonstrated through efforts like GSM8k, ProofNet, AlphaGeometry, and MathOdyssey. This progress has sparked interest in their potential use for tutoring students in mathematics. However, the reliability of LLMs in tutoring contexts -- where correctness and instructional quality are crucial -- remains underexplored. Moreover, LLM problem-solving capabilities may not necessarily translate into effective tutoring support for students. In this work, we present two novel approaches to evaluate the correctness and quality of LLMs in math tutoring contexts. The first approach uses an intelligent tutoring system for college algebra as a testbed to assess LLM problem-solving capabilities. We generate benchmark problems using the tutor, prompt a diverse set of LLMs to solve them, and compare the solutions to those generated by the tutor. The second approach evaluates LLM as tutors rather than problem solvers. We employ human evaluators, who act as students seeking tutoring support from each LLM. We then assess the quality and correctness of the support provided by the LLMs via a qualitative coding process. We applied these methods to evaluate several ChatGPT models, including 3.5 Turbo, 4, 4o, o1-mini, and o1-preview. Our findings show that when used as problem solvers, LLMs generate correct final answers for 85.5% of the college algebra problems tested. When employed interactively as tutors, 90% of LLM dialogues show high-quality instructional support; however, many contain errors -- only 56.6% are entirely correct. We conclude that, despite their potential, LLMs are not yet suitable as intelligent tutors for math without human oversight or additional mechanisms to ensure correctness and quality.

Momin N. Siddiqui, Adit Gupta, Jennifer M. Reddig et al.

Intelligent tutors have shown success in delivering a personalized and adaptive learning experience. However, there exist challenges regarding the granularity of knowledge in existing frameworks and the resulting instructions they can provide. To address these issues, we propose HTN-based tutors, a new intelligent tutoring framework that represents expert models using Hierarchical Task Networks (HTNs). Like other tutoring frameworks, it allows flexible encoding of different problem-solving strategies while providing the additional benefit of a hierarchical knowledge organization. We leverage the latter to create tutors that can adapt the granularity of their scaffolding. This organization also aligns well with the compositional nature of skills.