A New Method to Compare the Interpretability of Rule-based Algorithms
This provides a quantitative method for researchers and practitioners to evaluate interpretable algorithms, though it is incremental as it builds on existing metrics like predictivity.
The paper tackles the lack of consensus in measuring interpretability by proposing a score based on predictivity, stability, and simplicity to compare rule-based and tree-based algorithms, applying it to regression and classification cases.
Interpretability is becoming increasingly important for predictive model analysis. Unfortunately, as remarked by many authors, there is still no consensus regarding this notion. The goal of this paper is to propose the definition of a score that allows to quickly compare interpretable algorithms. This definition consists of three terms, each one being quantitatively measured with a simple formula: predictivity, stability and simplicity. While predictivity has been extensively studied to measure the accuracy of predictive algorithms, stability is based on the Dice-Sorensen index for comparing two rule sets generated by an algorithm using two independent samples. The simplicity is based on the sum of the lengths of the rules derived from the predictive model. The proposed score is a weighted sum of the three terms mentioned above. We use this score to compare the interpretability of a set of rule-based algorithms and tree-based algorithms for the regression case and for the classification case.