Exploring Conversational Design Choices in LLMs for Pedagogical Purposes: Socratic and Narrative Approaches for Improving Instructor's Teaching Practice
This addresses the challenge of adapting LLM interactions to support diverse instructors in education, though it is incremental in exploring specific conversational designs rather than introducing a new paradigm.
The researchers tackled the problem of designing effective conversational approaches for LLMs used in instructor professional development by comparing Socratic (guided questioning) and Narrative (elaborated suggestions) methods. They found that the Socratic version elicited greater engagement from 41 higher-education instructors, while the Narrative version was preferred for actionable guidance, with preferences varying by instructor experience and AI attitudes.
Large language models (LLMs) typically generate direct answers, yet they are increasingly used as learning tools. Studying instructors' usage is critical, given their role in teaching and guiding AI adoption in education. We designed and evaluated TeaPT, an LLM for pedagogical purposes that supports instructors' professional development through two conversational approaches: a Socratic approach that uses guided questioning to foster reflection, and a Narrative approach that offers elaborated suggestions to extend externalized cognition. In a mixed-method study with 41 higher-education instructors, the Socratic version elicited greater engagement, while the Narrative version was preferred for actionable guidance. Subgroup analyses further revealed that less-experienced, AI-optimistic instructors favored the Socratic version, whereas more-experienced, AI-cautious instructors preferred the Narrative version. We contribute design implications for LLMs for pedagogical purposes, showing how adaptive conversational approaches can support instructors with varied profiles while highlighting how AI attitudes and experience shape interaction and learning.