Interpreting Neurons in Deep Vision Networks with Language Models
This work addresses the need for better interpretability in deep vision networks, particularly for applications like land cover prediction in sustainability, though it is incremental as it builds on prior neuron description methods.
The paper tackles the problem of interpreting hidden neurons in deep vision networks by proposing Describe-and-Dissect (DnD), a novel method that uses multimodal deep learning to generate natural language descriptions without labeled data, resulting in higher quality labels and being over 2 times more likely to be selected as the best explanation than baselines.
In this paper, we propose Describe-and-Dissect (DnD), a novel method to describe the roles of hidden neurons in vision networks. DnD utilizes recent advancements in multimodal deep learning to produce complex natural language descriptions, without the need for labeled training data or a predefined set of concepts to choose from. Additionally, DnD is training-free, meaning we don't train any new models and can easily leverage more capable general purpose models in the future. We have conducted extensive qualitative and quantitative analysis to show that DnD outperforms prior work by providing higher quality neuron descriptions. Specifically, our method on average provides the highest quality labels and is more than 2$\times$ as likely to be selected as the best explanation for a neuron than the best baseline. Finally, we present a use case providing critical insights into land cover prediction models for sustainability applications. Our code and data are available at https://github.com/Trustworthy-ML-Lab/Describe-and-Dissect.