Foundations of Interpretable Models
This work addresses the foundational issue of interpretability definitions for AI researchers and practitioners, offering a more systematic approach to model design.
The paper tackles the problem of ill-defined interpretability in AI by proposing a new actionable definition that subsumes existing informal notions, and introduces a general blueprint and open-sourced library for designing interpretable models.
We argue that existing definitions of interpretability are not actionable in that they fail to inform users about general, sound, and robust interpretable model design. This makes current interpretability research fundamentally ill-posed. To address this issue, we propose a definition of interpretability that is general, simple, and subsumes existing informal notions within the interpretable AI community. We show that our definition is actionable, as it directly reveals the foundational properties, underlying assumptions, principles, data structures, and architectural features necessary for designing interpretable models. Building on this, we propose a general blueprint for designing interpretable models and introduce the first open-sourced library with native support for interpretable data structures and processes.