From unbiased MDI Feature Importance to Explainable AI for Trees
This work addresses interpretability issues for users of tree-based models, but it is incremental as it synthesizes and critiques existing approaches rather than introducing new methods.
The paper tackles the problem of interpretability and bias in tree-based models by unifying recent methods for debiasing Gini importance in random forests and connecting them to local explanation techniques, while also identifying a bias in existing explainable AI algorithms due to inbag data inclusion.
We attempt to give a unifying view of the various recent attempts to (i) improve the interpretability of tree-based models and (ii) debias the the default variable-importance measure in random Forests, Gini importance. In particular, we demonstrate a common thread among the out-of-bag based bias correction methods and their connection to local explanation for trees. In addition, we point out a bias caused by the inclusion of inbag data in the newly developed explainable AI for trees algorithms.