Making CNNs Interpretable by Building Dynamic Sequential Decision Forests with Top-down Hierarchy Learning
This addresses the need for interpretability in deep learning models, particularly for users in fields requiring transparency, though it is incremental as it builds on existing CNN and decision forest methods.
The paper tackles the problem of making CNNs interpretable without sacrificing accuracy by proposing a transfer scheme that builds a differentiable decision forest on top of CNNs, resulting in higher classification accuracy and better interpretability with plausible hierarchies and more precise saliency maps.
In this paper, we propose a generic model transfer scheme to make Convlutional Neural Networks (CNNs) interpretable, while maintaining their high classification accuracy. We achieve this by building a differentiable decision forest on top of CNNs, which enjoys two characteristics: 1) During training, the tree hierarchies of the forest are learned in a top-down manner under the guidance from the category semantics embedded in the pre-trained CNN weights; 2) During inference, a single decision tree is dynamically selected from the forest for each input sample, enabling the transferred model to make sequential decisions corresponding to the attributes shared by semantically-similar categories, rather than directly performing flat classification. We name the transferred model deep Dynamic Sequential Decision Forest (dDSDF). Experimental results show that dDSDF not only achieves higher classification accuracy than its conuterpart, i.e., the original CNN, but has much better interpretability, as qualitatively it has plausible hierarchies and quantitatively it leads to more precise saliency maps.