Activation Landscapes as a Topological Summary of Neural Network Performance
This provides a novel method for visualizing and analyzing neural network performance, which is incremental as it applies existing topological techniques to a new context in machine learning.
The paper tackles the problem of analyzing data transformations in deep neural networks by using topological data analysis to compute persistent homology of activation data, resulting in a feature map for visualization and statistical analysis, with observations that topological complexity often increases with training and does not decrease per layer.
We use topological data analysis (TDA) to study how data transforms as it passes through successive layers of a deep neural network (DNN). We compute the persistent homology of the activation data for each layer of the network and summarize this information using persistence landscapes. The resulting feature map provides both an informative visual- ization of the network and a kernel for statistical analysis and machine learning. We observe that the topological complexity often increases with training and that the topological complexity does not decrease with each layer.