Towards consistency of rule-based explainer and black box model -- fusion of rule induction and XAI-based feature importance
This work addresses the need for more reliable and consistent explanations in explainable AI (XAI) for users who rely on rule-based surrogates to understand complex models, though it is incremental as it builds on existing rule induction and XAI techniques.
The paper tackles the problem of ensuring that rule-based surrogate models used to explain black box models make decisions consistently with them, both in terms of outcomes and important attributes, by proposing a fusion method that integrates rule induction with XAI-based feature importance. The method was validated on 30 tabular benchmark datasets, showing improved consistency compared to a reference method.
Rule-based models offer a human-understandable representation, i.e. they are interpretable. For this reason, they are used to explain the decisions of non-interpretable complex models, referred to as black box models. The generation of such explanations involves the approximation of a black box model by a rule-based model. To date, however, it has not been investigated whether the rule-based model makes decisions in the same way as the black box model it approximates. Decision making in the same way is understood in this work as the consistency of decisions and the consistency of the most important attributes used for decision making. This study proposes a novel approach ensuring that the rule-based surrogate model mimics the performance of the black box model. The proposed solution performs an explanation fusion involving rule generation and taking into account the feature importance determined by the selected XAI methods for the black box model being explained. The result of the method can be both global and local rule-based explanations. The quality of the proposed solution was verified by extensive analysis on 30 tabular benchmark datasets representing classification problems. Evaluation included comparison with the reference method and an illustrative case study. In addition, the paper discusses the possible pathways for the application of the rule-based approach in XAI and how rule-based explanations, including the proposed method, meet the user perspective and requirements for both content and presentation. The software created and a detailed report containing the full experimental results are available on the GitHub repository (https://github.com/ruleminer/FI-rules4XAI ).