From Answer Generators to Reasoning Facilitators: Designing AI Tutors for Mathematical Reasoning in High-Stakes Environments
For designers of AI tutoring systems in high-stakes educational environments, this work provides empirical insights and a design framework (Reasoning-Centered Product Loop) to move beyond answer generation toward reasoning facilitation.
This paper presents AITutor, an interactive system designed to support mathematical reasoning for junior-high students preparing for high-stakes exams. Through a field deployment with 12 participants, the study reveals that students resist Socratic dialogue under time pressure and instead use 'answer-first' shortcuts as diagnostic checkpoints, with features like layered worked examples and metacognitive scaffolding lowering the cost of reasoning repair.
The rapid integration of Large Language Models (LLMs) into educational technology threatens to reduce mathematical learning to mere answer generation. This paper presents a generative study, usability study, and 12-participant field deployment of AITutor, an interactive system that translates theoretical pedagogical mechanisms into concrete user interface features. We explore how junior-high students preparing for high-stakes exams (Zhongkao) interact with AI tutoring. Through mixed-methods triangulation (7,379 telemetry events, 8 contextual observations, 10 interviews), we reveal that students actively resist traditional Socratic dialogue under time pressure, repurposing "answer-first" shortcuts as vital diagnostic checkpoints. We demonstrate how features like layered worked examples, step-linked visual grounding, and metacognitive scaffolding lower the interaction cost of reasoning repair. We contribute a "Reasoning-Centered Product Loop," offering actionable implications for designing AI that structurally supports the inspection, local repair, curriculum verification, and delayed retrieval of mathematical reasoning in the wild.