Comparing Deep Neural Nets with UMAP Tour
This work addresses the need for interpretability in neural networks for researchers and practitioners, though it is incremental as it builds on existing visualization techniques.
The authors tackled the problem of interpreting neural networks by developing UMAP Tour, a tool for visually inspecting and comparing internal layer behavior, which also led to a new similarity measure for layers, revealing learned concepts and differences in models like GoogLeNet and ResNet.
Neural networks should be interpretable to humans. In particular, there is a growing interest in concepts learned in a layer and similarity between layers. In this work, a tool, UMAP Tour, is built to visually inspect and compare internal behavior of real-world neural network models using well-aligned, instance-level representations. The method used in the visualization also implies a new similarity measure between neural network layers. Using the visual tool and the similarity measure, we find concepts learned in state-of-the-art models and dissimilarities between them, such as GoogLeNet and ResNet.