Measuring algorithmic interpretability: A human-learning-based framework and the corresponding cognitive complexity score
This provides a tool for managers to evaluate trade-offs in algorithm selection, addressing trust and fairness issues, but it is incremental as it builds on existing theories.
The authors tackled the lack of a formal measurement method for algorithmic interpretability by developing a framework based on programming language and cognitive load theories, resulting in a cognitive complexity score with properties like universality and computability.
Algorithmic interpretability is necessary to build trust, ensure fairness, and track accountability. However, there is no existing formal measurement method for algorithmic interpretability. In this work, we build upon programming language theory and cognitive load theory to develop a framework for measuring algorithmic interpretability. The proposed measurement framework reflects the process of a human learning an algorithm. We show that the measurement framework and the resulting cognitive complexity score have the following desirable properties - universality, computability, uniqueness, and monotonicity. We illustrate the measurement framework through a toy example, describe the framework and its conceptual underpinnings, and demonstrate the benefits of the framework, in particular for managers considering tradeoffs when selecting algorithms.