Promoting Critical Thinking With Domain-Specific Generative AI Provocations
This addresses the problem of enhancing critical thinking in knowledge work using AI, but it is incremental as it builds on existing ideas about AI-driven provocations.
The study tackled the mixed effects of generative AI on critical thinking by designing and evaluating two domain-specific tools, ArtBot for fine art interpretation and Privy for AI privacy, finding that domain-specific provocations with productive friction and user-dependent interactions can meaningfully support critical thinking.
The evidence on the effects of generative AI (GenAI) on critical thinking is mixed, with studies suggesting both potential harms and benefits depending on its implementation. Some argue that AI-driven provocations, such as questions asking for human clarification and justification, are beneficial for eliciting critical thinking. Drawing on our experience designing and evaluating two GenAI-powered tools for knowledge work, ArtBot in the domain of fine art interpretation and Privy in the domain of AI privacy, we reflect on how design decisions shape the form and effectiveness of such provocations. Our observations and user feedback suggest that domain-specific provocations, implemented through productive friction and interactions that depend on user contribution, can meaningfully support critical thinking. We present participant experiences with both prototypes and discuss how supporting critical thinking may require moving beyond static provocations toward approaches that adapt to user preferences and levels of expertise.