A review of possible effects of cognitive biases on the interpretation of rule-based machine learning models
It addresses the problem of human interpretability in machine learning for researchers and practitioners, but is incremental as a review transferring existing knowledge.
The paper reviews how cognitive biases affect human interpretation of rule-based machine learning models, covering twenty biases and debiasing techniques to bridge cognitive psychology and machine learning.
While the interpretability of machine learning models is often equated with their mere syntactic comprehensibility, we think that interpretability goes beyond that, and that human interpretability should also be investigated from the point of view of cognitive science. The goal of this paper is to discuss to what extent cognitive biases may affect human understanding of interpretable machine learning models, in particular of logical rules discovered from data. Twenty cognitive biases are covered, as are possible debiasing techniques that can be adopted by designers of machine learning algorithms and software. Our review transfers results obtained in cognitive psychology to the domain of machine learning, aiming to bridge the current gap between these two areas. It needs to be followed by empirical studies specifically focused on the machine learning domain.