Visualizing Deep Similarity Networks
This work provides visualization tools for similarity learning domains, addressing a need for interpretability in networks fine-tuned for image embeddings.
The authors tackled the problem of visualizing which image regions contribute most to pairwise similarity in convolutional neural networks optimized for image embeddings, by proposing a method that generalizes to different pooling strategies and supports similarity searches on objects or sub-regions.
For convolutional neural network models that optimize an image embedding, we propose a method to highlight the regions of images that contribute most to pairwise similarity. This work is a corollary to the visualization tools developed for classification networks, but applicable to the problem domains better suited to similarity learning. The visualization shows how similarity networks that are fine-tuned learn to focus on different features. We also generalize our approach to embedding networks that use different pooling strategies and provide a simple mechanism to support image similarity searches on objects or sub-regions in the query image.