Understanding Deep Neural Networks Using Topological Data Analysis
This addresses the interpretability problem for AI researchers and practitioners, but it appears incremental as it applies an existing method (TDA) to a new context in neural networks.
The paper tackled the problem of deep neural networks being black boxes by using Topological Data Analysis to analyze activation values in layers, providing insights into how the network processes validation images.
Deep neural networks (DNN) are black box algorithms. They are trained using a gradient descent back propagation technique which trains weights in each layer for the sole goal of minimizing training error. Hence, the resulting weights cannot be directly explained. Using Topological Data Analysis (TDA) we can get an insight on how the neural network is thinking, specifically by analyzing the activation values of validation images as they pass through each layer.