Deep Feature Factorization For Concept Discovery
This provides a method for gaining insight into what neural networks perceive, which is useful for researchers and practitioners in computer vision and AI interpretability.
The authors tackled the problem of interpreting deep convolutional neural networks by localizing similar semantic concepts in images, achieving state-of-the-art results in co-segmentation and co-localization tasks.
We propose Deep Feature Factorization (DFF), a method capable of localizing similar semantic concepts within an image or a set of images. We use DFF to gain insight into a deep convolutional neural network's learned features, where we detect hierarchical cluster structures in feature space. This is visualized as heat maps, which highlight semantically matching regions across a set of images, revealing what the network `perceives' as similar. DFF can also be used to perform co-segmentation and co-localization, and we report state-of-the-art results on these tasks.