Explaining the Predictions of Any Image Classifier via Decision Trees
This work addresses the need for better explainability in AI for users and developers, though it is incremental as it builds on the existing LIME framework.
The paper tackled the problem of explaining black-box image classifiers by proposing Tree-LIME, a decision tree-based method that replaces linear regression in LIME to capture nonlinear feature interactions, resulting in more reliable explanations with improved understandability, fidelity, and efficiency in experiments.
Despite outstanding contribution to the significant progress of Artificial Intelligence (AI), deep learning models remain mostly black boxes, which are extremely weak in explainability of the reasoning process and prediction results. Explainability is not only a gateway between AI and society but also a powerful tool to detect flaws in the model and biases in the data. Local Interpretable Model-agnostic Explanation (LIME) is a recent approach that uses an interpretable model to form a local explanation for the individual prediction result. The current implementation of LIME adopts the linear regression as its interpretable function. However, being so restricted and usually over-simplifying the relationships, linear models fail in situations where nonlinear associations and interactions exist among features and prediction results. This paper implements a decision Tree-based LIME approach, which uses a decision tree model to form an interpretable representation that is locally faithful to the original model. Tree-LIME approach can capture nonlinear interactions among features in the data and creates plausible explanations. Various experiments show that the Tree-LIME explanation of multiple black-box models can achieve more reliable performance in terms of understandability, fidelity, and efficiency.