Critical Inker: Scaffolding Critical Thinking in AI-Assisted Writing Through Socratic Questioning
This addresses the problem of reduced critical thinking skills for users of AI writing tools, representing an incremental improvement by integrating existing methods like Socratic questioning into a new system.
The paper tackles the risk of cognitive deskilling in AI-assisted writing by introducing Critical Inker, a tool that scaffolds critical thinking through Socratic questioning and visual feedback, achieving 91.2% argument overlap and 87% validity accuracy in evaluation.
As Large Language Models (LLMs) increasingly automate writing tasks, there is a growing risk of cognitive deskilling where users offload critical thinking to the system. To address this, we introduce Critical Inker, a writing tool designed to scaffold critical reflection during writing through logical analysis and socratic feedback. We present two methods: (1) A Socratic chatbot using questions to help them realize and fix logical errors in their writing and (2) Visual Feedback, which highlights logical errors in the text without dialog. We detail the technical implementation of the system and evaluate its argument extraction and logical validity accuracy. Our evaluation shows a 91.2% argument overlap with ground truth argument annotations and 87% validity accuracy. Finally, we conducted a small-scale pilot and discuss early qualitative results.