Intent Matters: Enhancing AI Tutoring with Fine-Grained Pedagogical Intent Annotation
This work addresses the need for better AI tutoring systems by improving response quality through intent annotation, though it is incremental as it builds on existing datasets and methods.
The study tackled the problem of aligning LLM-generated tutoring responses with pedagogical strategies by fine-tuning an LLM using a fine-grained annotation of teacher intents, resulting in more pedagogically aligned and effective responses as shown by automatic and qualitative evaluations.
Large language models (LLMs) hold great promise for educational applications, particularly in intelligent tutoring systems. However, effective tutoring requires alignment with pedagogical strategies - something current LLMs lack without task-specific adaptation. In this work, we explore whether fine-grained annotation of teacher intents can improve the quality of LLM-generated tutoring responses. We focus on MathDial, a dialog dataset for math instruction, and apply an automated annotation framework to re-annotate a portion of the dataset using a detailed taxonomy of eleven pedagogical intents. We then fine-tune an LLM using these new annotations and compare its performance to models trained on the original four-category taxonomy. Both automatic and qualitative evaluations show that the fine-grained model produces more pedagogically aligned and effective responses. Our findings highlight the value of intent specificity for controlled text generation in educational settings, and we release our annotated data and code to facilitate further research.