Reversing the Lens: Using Explainable AI to Understand Human Expertise
This work addresses the challenge of quantitatively studying human cognition in safety-critical domains, offering a novel application of XAI to bridge psychology and AI, though it is incremental in combining existing techniques.
The study tackled the problem of understanding human expertise in complex tasks by applying Explainable AI (XAI) methods to analyze operator behavior in tuning a particle accelerator, revealing how operators decompose problems and evolve strategies with experience.
Both humans and machine learning models learn from experience, particularly in safety- and reliability-critical domains. While psychology seeks to understand human cognition, the field of Explainable AI (XAI) develops methods to interpret machine learning models. This study bridges these domains by applying computational tools from XAI to analyze human learning. We modeled human behavior during a complex real-world task -- tuning a particle accelerator -- by constructing graphs of operator subtasks. Applying techniques such as community detection and hierarchical clustering to archival operator data, we reveal how operators decompose the problem into simpler components and how these problem-solving structures evolve with expertise. Our findings illuminate how humans develop efficient strategies in the absence of globally optimal solutions, and demonstrate the utility of XAI-based methods for quantitatively studying human cognition.